Go! Speed Racer Go!

I finally reached a point where I could start running the go version of sm-photo-tool. I finished the option validation for the list command. While I was testing it I noticed how much faster the Go version felt. Here are the python vs Go versions of the commands.

The python version took 80 milliseconds to run the validation of parameters and print out the error message. This is expected, you have to start the python VM (and if not precompiled, the code would need to be compiled). In this test, the code was precompiled.

$ time sm-photo-tool list invalid blah
ERROR: valid options are ['album', 'galleries']

real	0m0.080s
user	0m0.059s
sys	0m0.021s

Ok so how fast is the Go version? My guess was half the time, 40ms. I was way off. Try 6ms. SIX! That’s amazing to do the exact same amount of work.

$ time ./sm-photo-tool list invalid blah
ERROR: valid options are [album galleries]

real	0m0.006s
user	0m0.001s
sys	0m0.005s

Private Golang, reporting for duty

Many modern programming languages have a mechanism for controlling the visibility of members and methods. Most of the languages use keywords like private and public.

Java and C++ use public, private, and protected. Ruby also uses keywords: private, and protected.

Python on the other hand uses convention to control visibility, prefixing your methods with _ or __. Even then it’s more of a suggestion in Python as you can call it anyway. There is nothing to prevent it from actually being called from other code. It’s an understanding among Python developers that if there is an underscore you shouldn’t call it.

Go also has no keyword to control visibility. What no keyword? There has to be something? Maybe it uses _ like python? Nope that won’t do anything. Then what would Go use to control visibility? Go uses a very simple syntax, capitalization.

CAPITALIZATION? What? Yes, that’s correct. A simple capital letter makes the member or method public. Lowercase member, methods are private.

// GetName is a public method of User struct
func (u *User) GetName() string {
  return u.name
}

// nickName is a private method of User struct
func (u *User) nickName() {
  // do something
}

The same logic applies to members. Given the following example struct, Foo is public and bar is private:

type Example struct {
  Foo string
  bar string
}

So if you want to make your Go methods or members private, just start them with a lowercase letter. Want others to use it, capitalize it.

Adventures with the Go programming language

After 10+ years of working with Java, I spent the last year working with Ruby & Rails. Let’s just say I’m not a fan. I wanted to learn something new and I’ve heard a lot of good things about Go. Quite a few projects use Go: Docker, Hugo, Kubernetes, Prometheus, and many more.

Three weeks ago, I picked up a copy of “The Go Programming Language” by Donovan & Kernighan. Yes, the same Kernighan that co-wrote the famous, “The C Programming Language”.

In order to learn it, I needed a real life scenario, I needed a project. Why not port, sm_photo_tool, my Smugmug command line tool from python to Go lang? That will give me experience with how Go handles parsing arguments, classes, REST APIs, concurrent uploads, and a bunch more things.  It’s the perfect project to learn Go.

In the next post I’ll talk about structs in Go.

Fedora 13 to Centos 7

A couple of weeks ago, I finally updated my home server from Fedora 13 to CentOS 7. Fedora had the media support but I can’t keep the machine updated fast enough. This time I decided to go with another Red Hat derivative so I chose CentOS. The goal is to have an OS that has a longer release cycle. Most of the services I’m running haven’t changed in a while (considering Fedora 13 was ancient).

This was NOT an in place “upgrade”. Going to CentOS was a fresh install. I backed up all 70G of the root partition to an external drive then told CentOS to reformat the existing partitions. My biggest worry was the 2TB RAID 5 array. During the installation I told the installer to mount the array on /vol but DO NOT FORMAT. I checked it three times to make sure I didn’t lose anything. I did NOT backup the RAID array so this was the most dangerous part of the install I did.

Once installed the task of restoring the data back to the server, users, configuration, install needed software to support the services I had running on the machine, etc. Because Fedora 13 was so old, I couldn’t just rsync /etc.

First thing I had to do was change the UID and GUID of all the backup files since the default UID changed from 500 to 1000. I created the users, then I had to change the access to the backed up files and to the RAID array.

# owner
chown -R --from=500 1000 $1
chown -R --from=501 1001 $1
chown -R --from=502 1002 $1
chown -R --from=504 1004 $1
chown -R --from=505 1005 $1
chown -R --from=506 1006 $1
chown -R --from=507 1007 $1

# groups
chown -R --from=:500 :1000 $1
chown -R --from=:501 :1001 $1
chown -R --from=:502 :1002 $1
chown -R --from=:504 :1004 $1
chown -R --from=:505 :1005 $1
chown -R --from=:506 :1006 $1
chown -R --from=:507 :1007 $1

chown -R --from=481:466 992:989 $1

The firewall switched from iptables to FirewallD. So instead of restoring my /etc/sysconfig/iptables file I decided to figure out how to use FirewallD. I wrote a script for each of the iptables ports:

#...
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-port=PORT#/tcp
#...
firewall-cmd --reload

Most of the services were easy to get back running: http, vnc, and plexmediaserver. Some of the others were a bit tricky to get back up and running.

Streambaby had an error with the newer version of ffmpeg. It would fail with:

Exception in thread "main" java.lang.NoClassDefFoundError: net/sf/ffmpeg_java/v53/AVFormatLibrary …

Thankfully, this post had the suggested solution of simply adding the following to the streambaby.ini

com.unwiredappeal.tivo.vm.ffjava.FFmpegJavaVideoModule=false
ffmpegexe.transcode.sameqargs=-ab 192k

Other Streambaby & multimedia resources:

https://code.google.com/p/streambaby/wiki/StreamBabyIni
http://info.vortexbox.org/tiki-index.php?page=InstallStreamBaby
http://wiki.centos.org/TipsAndTricks/MultimediaOnCentOS7

Next up was to revive the NFS shares.

The server also acts as a TimeMachine drive to backup our MacBook Pro. The old server used netatalk 2.x so the configuration files changed here too.

  • config files moved from /etc/atalk to /etc/netatalk
  • afpd.conf became afp.conf
  • afp conf is completely different

I basically followed the following sources:
http://netatalk.sourceforge.net/wiki/index.php/Netatalk_3.1.7_SRPM_for_Fedora_and_CentOS
http://netatalk.sourceforge.net/3.1/htmldocs/configuration.html#idp139639181431264

The last service was to setup VirtualBox to run headless for my Windows 7 guest. See VirtualBox installation it wasn’t that difficult.

That’s was pretty much the only things I hit during the “upgrade”. It took a couple days to get everything up and running. I did it one service at a time.

One lesson I learned? Don’t wait so long to update your servers 🙂

Candlepin 0.5.5 released

Candlepin

It’s that time again, another release of Candlepin and associated projects available for your enjoyment. With this release we have subscription-manager in Fedora as well as a debut build of Thumbslug.

For more information on Candlepin, please visit: http://candlepinproject.org/

Features & Enhancements

subscription-manager

  • added support for host registration and guest association when host can not register itself
  • virt-who work to handle ESX guests
  • subscription-manager available in fedora

candlepin

  • build modified to use the tito hotness instead of bunch of disjoint bash scripts
  • disable manifest rules import
  • added support for host registration and guest association when host can not register itself

thumbslug

  • added appropriate init scripts to run as a service
  • uses Candlepin CRL
  • thumbslug talks to akamai
  • created puppet module for katello

Bugs fixed

subscription-manager

705883 Fix error dialog modal issues.
719743 Improved text output for successful pool subscription
740788 Getting error with quantity subscribe using subscription-assistance page.
746259 Don’t allow the user to pass in an empty string as an activation key
746732 Only use fallback locales for dates we need to parse
749332 Normalize the error messages for not being registered
749636 Client should not support users entering activation keys and existing consumer ids
752572 add interval logging statements back in on rhsmcertd startup
753093 The available subscriptions count does not show correctly in Subscription Manager GUI
754821 Default org of “Unknown” was not marked for gettext
755031 Unregister before attempting to run a second registration
755035 Migration script should work on RHEL 5.7 and up.
755130 add extra whitespace to classic warning
755541 Enhanced the message in the katello plugin to debug when the backend system does not support environments.
756173 Unexpected behavior change in subscription-manager unregister
756507 do not use output from “getlocale” as input for “setlocale”
758471 install-num-migrate-to-rhsm threw traceback when no instnum was found.
759199 rhsmcertd is logging the wrong value for certFrequency

candlepin

753093 The Available Subscriptions count do not show correctly in Subscription Manager GUI
754841 Implement DELETE /pools/id.
754843 Fix legacy virt bonus pools missing pool_derived.
755677 Activation Keys should not check quantity on unlimited pools
756628 Translate missing rule errors.
758462 ensure job detail isn’t null, skip it.

thumbslug

759607 update url for subscriptions handler

Download & Setup

Make sure you read over the Candlepin Setup Guide, which is located at https://fedorahosted.org/candlepin/wiki/Setup.

As well as the Headpin Install Guide which can be found at https://fedorahosted.org/candlepin/wiki/headpin/Install

Just give me the bits already! You can get the various bits at the urls below.

Candlepin:
http://repos.fedorapeople.org/repos/candlepin/candlepin/

Thumbslug:
http://repos.fedorapeople.org/repos/candlepin/thumbslug/

Headpin:
http://repos.fedorapeople.org/repos/katello

Subscription Manager:
http://repos.fedorapeople.org/repos/candlepin/subscription-manager/

Candlepin 0.4.16 released.

Another sprint gone by and another release of Candlepin for your enjoyment. Candlepin 0.4.16 is ready. You can get the bits at:

http://repos.fedorapeople.org/repos/candlepin/candlepin/

Make sure you read over the Setup Guide, which is located at
https://fedorahosted.org/candlepin/wiki/Setup

For more information on Candlepin, please visit our project page.

Features & Enhancements

client

  • A number of GUI changes
    • Center the machine type column header
    • Move quantity column to the end
    • Center the Arch column header
    • Center tree view table properties
    • Add ‘* Click to Adjust Quantity’ label to places allowing editable subscription quantity
    • New icons for red/green
    • Add virt_only attribute to subscription detail pane
    • Display subscription assistant’s subscriptions as a tree
    • Double click or button press (enter, return, space) on row will expand/collapse row
    • Update to All Available Subscriptions tab to put stacked subscriptions under parent node
    • Moved multi-entitlement column (*) next to the quantity column
    • Made the contract selector a little wider so all columns were visible (no manual resize)
  • Initial work done for the healing feature
    • Changes to rhsmcertd to support healing frequency (part I)
    • Add autoheal option to certmgr.py
    • Only autoheal when required
    • Use server-side consumer autoheal flag
  • Misc items
    • Update the strings and the remote server location
    • Make “make stylish” run all the checks, make whitespace “pop”
    • Update translations
    • managerlib was expecting a single ent_cert, but we return a list
    • Add a “refresh” method to cert_sorter
    • Add a require_connection callback to commands

server

  • upgraded to RESTEasy 2.2.1GA
  • export virt entitlements to non-candlepin consumers
  • refactored pinsetter to work in clustering mode
  • add new api to query jobs by owner, principal, consumer uuid

Bugs fixed

707641 CLI auto-subscribe tries to re-use basic auth credentials
712047 yum prints non-error messages when running in quiet mode
718052 Remove owner from consumer resource return codes. Only use the term org.
730020 Change the help text to show that config can list or set changes
731577 API to query jobs by owner, principal, consumer uuid.
731996 SQL Error when using REST query for events
732538 Disallow the relationship between a ‘person’ pool and an activation key
734174 Add missing produces annotations for role resource.
734880 Handle bundled certs in the installed produict status.
734606 ImportFileExtractor now creates cert/key files based on serial number of the cert
735087 If quartz is in clustered mode, we shouldn’t schedule any jobs.
735226 Importing should fail without a valid key and cert
735338 Subscription Manager CLI tool does not allow unsubscribe when not registered.
735695 add support for multiple config “–remove” options via cli
736166 move certs from subscription-manager to python-rhsm
736784 config –remove add config property to rhsm.conf if it doesn’t exist.
737841 Handle dates beyond 2038 on 32-bit systems.