mylist => foo:bar:baz

My module needed to query the server and determine if the list of things in that fact matched its list from hiera.

1
2
3
4
5

my_list:
      - 'foo'
      - 'bar'
      - 'baz'
      - 'qux'

Puppet would see qux as missing run the steps necessary to add it to the server. On the next run, the fact contains qux so no more work is needed.

I spent some time working on variations of regular expression matching, but could never get it to work.

1
2
3
4
5
6

if "${::my_list}" =~ /$word/ {
  notify { "${::my_list} has ${word} in it":}
}
else {
  notify { "${::my_list} doesn't have ${word} in it":}
}

After some research I found what I think was the issue.

Regular expressions cannot be converted to boolean values.

Luckily, puppetlabs/stdlib provides a clean way to work around that. Using the member function I split my_list into an array, and then test if word is part of that array.

1
2
3
4
5
6

$my_array = split($::my_list, ':')
if member($my_array, $word) {
  notify { "${my_array} has ${word} in it":}
} else {
  notify { "${my_array} doesn't have ${word} in it":}
}

That is not going to happen. Sorry. There are already resources out there to teach puppet, and they are excellent. A module I write in a day shouldn’t take weeks to write about. At my current pace that is what is going to happen. Instead, I’ll just describe what the module does and post the code.

Subversion Module

• Installs Subversion
• Creates repository directory
• Adds a facter fact to list the current repositories on the server
• Uses Hiera for crowd specific variables and the repositories on the server

Puppetlabs Apache Module

• Expanded to support auth_crowd

Code

Code for the subversion module can be found here. I’ll post the puppetlabs-apache code once I submit a pull request with my additions.

• Install subversion
• Install apache
• Install Atlassian crowd apache connector (mod_authnz_crowd)

The first module will be straight forward, install a package for subversion. It will get more difficult after that because I plan on using the puppetlabs/apache module. I will either need to expand their module for mod_authnz_crowd support, or write around the httpd configuration files it manages.

I try to use the same directory template for all of my modules. It looks like this:

1
2
3
4
5
6

template/
|-- files
|-- lib
|-- manifests
|-- templates
`-- tests

There is also a function in the puppet tool to create a new module using their standard, but I haven’t had luck with it. I’m going to use my directory template to create a simple subversion module.

One of the things I find helpful is to document the goals of my modules before I write any code. In this case I have a general idea of what is required because I built a test system and configured everything on it. For subversion, I need to do the following (at least):

• Install subversion
• Create repository directory

Here is the code I wrote to do those things:

1
2
3
4
5
6
7
8
9

class subversion (
  $repository_dir       = [ "/var/", "/var/www/", "/var/www/svn/", ],
  $package_name         = 'subversion',
  ) {

    include subversion::install
    include subversion::repository

  }

1
2
3
4
5

class subversion::install {
  package {  "${subversion::package_name}":
    ensure => installed,
  }
}

1
2
3
4
5

class subversion::repository {
  file { $subversion::repository_dir:
    ensure => "directory",
  }
}

The basic module works and accomplishes the goals listed above:

• Install subversion

1
2
3

[root@burner ~]# rpm -qa | grep -i subversion
subversion-1.6.11-7.el6.x86_64
[root@burner ~]#

• Create repository directory

1
2
3
4
5

[root@burner ~]# ls -larh /var/www/svn/
total 8.0K
drwxr-xr-x. 3 root root 4.0K Mar 29 15:49 ..
drwxr-xr-x. 2 root root 4.0K Mar 29 15:49 .
[root@burner ~]#

But, it leaves a lot to be desired and some things to think about:

• Aren’t all the cool kids using hiera now?
• There is a directory for repositories there, should I add repository creation to the module?
• Should the subversion module be expanded to install apache?

1

class {'apache::mod::mod_authnz_crowd': }

• Or, should the apache module be expanded to install subversion?

1

class {'subversion::mod_authnz_crowd': }

• Or something completely different?

Today I had a task to get crowd integrated with Subversion for authentication and authorization. This was not a huge hurdle, Atlassian provides an RPM for their crowd apache connector and the setup was straight-forward. They do not at this time provide an RPM repository (or my half hearted search didn’t turn up anything).

I don’t like doing one off installations any more. I would much rather use puppet to perform installation and management. The path of least resistance would be to use the rpm provider to install and manage the RPM. But, long term, I would much rather have an internal repository

Enter Pulp

There are many ways to accomplish setting up an RPM repo, but Pulp seems to be the way that RHEL is moving.

First impressions

Because this is the first time I’ve used Pulp, some initial impressions. Keep in mind, this is with pretty much zero actual usage of the software.

• Installation is easy
• The documentation about which RPMs to install is a little off
• Holy hell, it uses an AMPQ messaging engine
• Aaaand, there seems to be an agent to install on “consumers”

As an aside, I’m getting annoyed with agents on servers. I understand why they are needed, enjoy the benefits, but hate the sprawl. Typical servers can have agents for:

• monitoring
• backup
• puppet
• mcollective
• file integrity monitoring (tripwire, etc..)
• package management?

First repository

Getting RHEL added into Pulp was easy, which I wasn’t expecting considering you have to go through a server subscription process. I followed the instructions here.

1
2
3
4
5
6
7

subscription-manager register
subscription-manager refresh
subscription-manager subscribe --auto

pulp-admin rpm repo create --id=rhel-6-server --feed=https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os --feed_ca=/etc/rhsm/ca/redhat-uep.pem --feed_key=/etc/pki/entitlement/longnumber-key.pem --feed_cert=/etc/pki/entitlement/longnumber.pem

pulp-admin rpm repo sync run --repo-id=rhel-6-server

And it started syncing… slowly. For the future, doing something like this seems to increase the download workers.

1

pulp-admin rpm repo update --num-threads=50 --repo-id=rhel-6-server

Private repository

Getting RHEL to sync is nice, but what I need for my crowd project is a private repo where I can throw the RPM.

1
2

pulp-admin rpm repo create --repo-id=customrepo
pulp-admin rpm repo uploads rpm --repo-id=customrepo  -f mod_authnz_crowd-2.0.1-1.x86_64.rpm

Ignoring the client.. for now

The client and back end messaging engine look really great, despite what I said earlier. But I want to get a good feel for pulp before deploying an agent to all my machines. First I tried just adding the repo to /etc/yum.repos.d:

1
2
3
4
5
6
7

[root@testserver yum.repos.d]# cat private.repo 
[customrepo]
name=Custom Repository
baseurl=https://pulp.private.lan/pulp/repos/customrepo
enabled=1
gpgcheck=0
[root@testserver yum.repos.d]#

But that complained with an error about certificates. My initial guess is the server wants the pulp agent to register with it.

1
2

[Errno 14] Peer cert cannot be verified or peer cert invalid
Trying other mirror.

My work around was simply to remove the https requirement from the repo.

1
2

pulp-admin rpm repo update --repo-id=customrepo --serve-http=true
pulp-admin rpm repo publish run --repo-id=customrepo

Switched the repo on the server to http and it’s able to communicate and pull the RPM I uploaded.

Things left to do

That gets me to the point of working on the puppet module for crowd, but pulp is a much larger project than I expected so I have more to do.

• Determine if the certificate error is due to requiring the agent
• Explore the features of the pulp agent to see if it’s worth pursuing
• Add the RHEL repository I created to a test machine and play with using it instead of Red Hat
• Setup automatic syncing of repositories
• Switch back to SSL
• Add the Red Hat “optional” channel as synchronized repositories
• Play with adding a puppet module repository and figure out how that impacts my current version controlled release pattern

More I’m sure, but that’s a good start.

I created a skeleton configuration for managing multiple locations in puppet and pushed it to github. You can find the repository here.

Two sites are defined, Earth and Mars. Parameters for each site are defined in site/manifests/earth/params.pp and site/manifests/mars/params.pp. Included are two modules ntp and user to demonstrate configuring modules across sites.

I also sent a message to puppet-users to get feedback on the methods I used. If I hear improved ways to do this type of setup I’ll update the repository.

$ sudo rhn-channel --add --channel=rhel-x86_64-server-optional-6
$ sudo yum install http://pkgs.repoforge.org/git/git-1.7.11.1-1.el6.rfx.x86_64.rpm http://pkgs.repoforge.org/git/perl-Git-1.7.11.1-1.el6.rfx.x86_64.rpm

The annoying part to this was finding the rhn-channel command to enable the optional repo. Most places I looked online wanted me to do this

yum --enablerepo=rhel-6-server-optional-rpms

but my RHEL server wasn’t subscribed to that channel for it to be enabled.

If you think so too, let me know and we’ll code some stuff up! Or, if this already exists and I haven’t heard of the site let me know.

Begin Elevator Pitch (in car salesman like voice):

Achievements, the “gamification” of the web, are poised to become the next big thing. Microsoft even recently added earning achievements to the next release of Visual Studio.

I propose a centralized place, much like gravatar, that would allow developers to easily add achievements to their site. Users benefit because they have one place to keep track of all their “badges” and unlockable content.

(all of these based on user and site preference)

• Site can administrators easily add and track new achievements by using API or portal
• User achievements/badges could be shared among sites
• Posting of newly earned badges across all sites to twitter/facebook/etc..
• Linked to email address, no real “authentication” needed
• Increase site visibility because badges for your site would show up on other site user profiles
• Increase user interactions as they compete for badges

I think the idea is fun, and I’ve been able to pull down an Ubuntu ISO using only DNS as my source. I did a call for help on reddit for people to give suggestions on cleaning up my code and already have four pull requests. Still have quite a few things I’d like to add, at the top of my list is a rails app that would allow a user to upload a file and have it pushed to DNS records.

Check it out, make fun of my code, and tell anyone you think might be interested.

batch_incrementor = 0
batch_size = 10

50.times do |i|

  fork do
    puts i
    sleep 10
  end
  
  batch_incrementor += 1
  
  if (batch_incrementor/batch_size.to_f)%1 == 0.0
    Process.waitall
  end
  
end

I’m sure this method isn’t original or new, just new to me.

If any real programmers read this and have better ways, please let me know! I’m always ready to learn more.

From the Polycom SIP 2.0 Administrator’s Guide

If set to 1, use volatile storage for phone-resident copy of the directory to allow for larger size.

Keep in mind that all of these settings are wrapped in XML. Don’t remove any of that formatting, just change these values in the XML.

Before they were changed in sip.cfg

dir.local.volatile.2meg="0"
dir.local.volatile.4meg="0"
dir.local.volatile.8meg="0"

After they were changed in sip.cfg

dir.local.volatile.2meg="1"
dir.local.volatile.4meg="1"
dir.local.volatile.8meg="1"

A coworker and I had a discussion over lunch that had me thinking about digital privacy. A digital profile is the aggregate information that is collected about your online activities. What our “digital profile” says about us and what it can be used for is both entertaining and scary. The entertaining part is coming up with fictional scenarios that could happen, the scary part is that the scenarios could be happening now.

Cue twilight zone narration voice

Scenario 1:
A health insurance company is trying to lower rates for customers. By profiling customer’s search history and social network information they can more accurately place people into high or low risk categories and determine what rates they should pay. For customers that have opted for a *private* social network, information is pulled from their “Friends” who have public data and a profile is built based upon aggregate friend information. As the program continues, more information is gathered about the users. Suddenly someone who posts something supporting breast cancer awareness is in a higher risk category because of an increased probability they are related to someone with cancer.

Scenario 2:
A car insurance company is trying (again) to lower rates for customers. They use the same methods above but also found that with cell phone GPS they can accurately take snapshots of how fast the customer is driving. Suddenly someone who has been a safe driver under the previous system that drives 10 miles over the speed limit is bumped to a riskier category.

It easy to come up with scenarios once you get started:

• Hiring (or firing)
• Getting added to a “no fly list”
• Loan applications
• Legal cases (think divorce, paternity)
• Identity theft
• Benefits claims (based on some activity you post, contesting eligibility for benefits)

So should you try to hide your identity on the Internet? Personally, I would rather have privacy laws in place that protect user data. I don’t want a world where only sophisticated users can protect their information. I don’t want to hide in an Internet cave battling new ways to collect information about me. I enjoy using the new social services that are coming out. Perhaps I’m naive.

Some links to check out:

• EFF
• Google’s upcoming privacy policy
• Facebook privacy policy
• International Safe Harbor Privacy Principles

# Free SIP extensions
# Usage: freesip /etc/asterisk/sip.conf 200 299

freesip() {
  comm -2 <(seq $2 $3) <(cat $1 | grep ^\[ | sort | uniq | tr -d [ | tr -d ]) | grep ^[[:digit:]]
}

$ freesip /etc/asterisk/sip.conf 200 299
200
201
233
245
$