0

Alternate Content Sources and You (or How to rebuild your Satellite and not have to download all the content from the CDN again)

Given that I have some downtime from work, I wanted to make some serious changes to my lab.

In my lab currently, I have a system cdn.auroracloud.com, which functions as my ‘big dumb repository server’, that I use to mirror Red Hat content so that I can make it available to other systems & Satellite. The challenge that I am dealing with is that this system is running very low on disk space. And as such, I want to move it to new hardware. Normally, this is a simple operation of using the katello-backup and katello-restore commands to backup and restore the data somewhere else with bigger disks. However, this Satellite has had a number of marginally supported tweaks and some borderline criminal hacks applied to it, and I rather start off with a fresh installation. (Plus this is my lab, so I don’t have old data I need to maintain). But that does leave one pretty major problem:

How do I preserve all of these RPMs that I have previously downloaded?

According to the df command, have a good deal north of 400G in use:

[root@cdn:] ~ #df
Filesystem                 Type      Size  Used Avail Use% Mounted on
/dev/mapper/vg_osdata-root xfs       498G  463G   36G  93% /

Enter Alternate Content Sources, which is a means to tell a Pulp server (such as the one included in Satellite 6) to load content from a alternate source (hence the name), allowing me to effectively sideload all of the RPMs on my new install.

My workflow in a nutshell is:

  • Export all of the content from the source Satellite to a directory.
  • Make that directory available via a generic web server
  • Build the destination Satellite.
  • Configure the destination Satellite to use the alternate content source
  • sync.

Exporting content.

On the source Satellite, I need a means to export the content in a manner suitable for reimport. The general method for exporting content is using Inter-Satellite Sync, but I am lazy and don’t feel like waiting on Pulp to actually export the data. And since the data is already on disk in the right directories, I can simply use the rsync command to export it.

On the source Satellite, I export the content to my webserver 192.168.1.11

# rsync -harv -e ssh -L /var/lib/pulp/published/yum/https/repos/<YOUR_ORG>/Library/content root@192.168.1.11/var/www/html/pub/content

The magic here is the -L | --copy-links option to rsync which (as per its man page): transform symlink into referent file/dir. This option basically undoes the fact that Pulp does single-instance storage of RPMs, and allows the export to work, without broken symlinks. This does come at the expense of additional disk space consumption.

Making the content available

On my webserver, which is a generic system running Apache, I’ve made the /var/www/html/pub/content directory available via http://192.168.1.11/pub/content

Build the Destination Satellite.

I won’t bore you with the details, but I’ve built the new destination Satellite as per the Installation Guide

Configuring the destination Satellite to use the alternate content source

This is the fun part. Now that I have the content available via HTTP. I need to configure the destination Satellite to use it. According to the
docs, I need to create a file in /etc/pulp/content/sources/conf.d point to the location of the alternate content source and each directory which contains a repodata subdirectory. Knowing that I have a ton of repositories, cataloging these by hand would suck. Though with a little bit of python, this is stupid simple. (as XKCD shows us)

#!/usr/bin/env python
import os

DIRECTORY='/var/www/html/pub/content/'

print "[pulp-content-source]"
print "enabled: 1"
print "priority: 0"
print "expires: 3d"
print "name: Pulp Content Source"
print "type: yum"
print "base_url: http://192.168.1.11/pub/content/"
print "paths:",

for root, directories, filenames in os.walk(DIRECTORY,followlinks=True):
  for directory in directories:
    if directory == 'repodata':
      RELATIVE_DIR = os.path.join(root, directory).replace(DIRECTORY,'').replace('repodata','')
      print "  " + RELATIVE_DIR

Copying this script and running this on my web server nets me the following, which I saved on the destination Satellite as /etc/pulp/content/sources/conf.d/local.conf:

[pulp-content-source]
enabled: 1
priority: 0
expires: 3d
name: Pulp Content Source
type: yum
base_url: http://192.168.1.11/pub/content/
paths:   beta/rhel/server/7/x86_64/satellite/6/os/
  eus/rhel/server/7/7.3/x86_64/os/
  dist/rhel/client/6/6.2/x86_64/kickstart/
  dist/rhel/client/6/6.2/x86_64/kickstart/Client/
  dist/rhel/client/6/6.2/x86_64/os/
  dist/rhel/client/6/6.8/i386/kickstart/
  dist/rhel/client/6/6.8/i386/kickstart/Client/
  dist/rhel/workstation/7/7Workstation/x86_64/os/
  dist/rhel/workstation/7/7Workstation/x86_64/sat-tools/6.2/os/
  dist/rhel/workstation/7/7.2/x86_64/kickstart/
  dist/rhel/server/7/7.1/x86_64/kickstart/
  dist/rhel/server/7/7.1/x86_64/kickstart/addons/HighAvailability/
  dist/rhel/server/7/7.1/x86_64/kickstart/addons/ResilientStorage/
  dist/rhel/server/7/7.1/x86_64/os/
  dist/rhel/server/7/7.3/x86_64/kickstart/
  dist/rhel/server/7/7.3/x86_64/kickstart/addons/HighAvailability/
  dist/rhel/server/7/7.3/x86_64/kickstart/addons/ResilientStorage/
  dist/rhel/server/7/7Server/x86_64/rhvh-build/4/os/
  dist/rhel/server/7/7Server/x86_64/rh-common/os/
  dist/rhel/server/7/7Server/x86_64/rhvh/4/os/
  dist/rhel/server/7/7Server/x86_64/jbeap/7/os/
  dist/rhel/server/7/7Server/x86_64/os/
  dist/rhel/server/7/7Server/x86_64/sat-capsule/6.2/os/
  dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os/
  dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/os/
  dist/rhel/server/7/7Server/x86_64/rhv/4.0/os/
  dist/rhel/server/7/7Server/x86_64/optional/os/
  dist/rhel/server/7/7Server/x86_64/rhscl/1/os/
  dist/rhel/server/7/7Server/x86_64/rhv-mgmt-agent/4/os/
  dist/rhel/server/7/7Server/x86_64/rhevh/os/
  dist/rhel/server/7/7Server/x86_64/supplementary/os/

<-- OUTPUT SNIPPED FOR BREVITY -->

Next, let’s make source our configuration is sane. To do this, we need the pulp-admin command.

# yum install -y pulp-admin-client

Store the pulp server password as a variable, as we’ll need it for future commands

# PULP_PASS=$(grep ^default_password /etc/pulp/server.conf | cut -f 2 -d ':')

And now let’s ensure that pulp sees our content source correctly:

# pulp-admin -u admin -p $PULP_PASS content sources list
+----------------------------------------------------------------------+
                            Content Sources
+----------------------------------------------------------------------+

Base URL:       http://192.168.1.11/pub/content/
Enabled:        1
Expires:        3d
Max Concurrent: 2
Name:           Pulp Content Source
Paths:          beta/rhel/server/7/x86_64/satellite/6/os
                eus/rhel/server/7/7.3/x86_64/os
                dist/rhel/client/6/6.2/x86_64/kickstart
                dist/rhel/client/6/6.2/x86_64/kickstart/Client
<-- OUTPUT SNIPPED FOR BREVITY -->
Priority:       0
Source Id:      pulp-content-source
SSL Validation: true
Type:           yum

The pulp-admin command will bark very loudly if the configuration is incorrect.

Sync

Now that we have our new destination Satellite configured to use our alternate content source. All that is left is to synchronize some repositories. Sync some repos using any normal method. You can confirm that your content source is working by:

  • checking via journalctl. You’ll see messages such as Dec 21 11:56:20 cdn.auroracloud.com pulp[21825]: nectar.downloaders.threaded:INFO: Download succeeded: http://192.168.1.11/pub/content/dist/rhel/server/6/6Server/x86_64/optional/os/libpurple-perl-2.6.6-6.el6_0.x86_64.rpm.
  • checking the access logs of your webserver.

You’d see that the Satellite server will still get metadata from the Red Hat CDN (to understand which RPMs are in the repository), however, it will prefer the alternate content source for RPMs that match (and are available there).

Rich Jerrido

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.