Pretty Print directory of .json files

I had a bunch of compressed json files that I needed to pretty print to make them more readable. This little snippet will create a new pretty printed json file prefixed with pp:

ls *.json | xargs -I {} sh -c "cat {} | python -mjson.tool > pp{}"

Instead of having to look at files that look like this:

{ "attributes": [ { "name": "type", "value": "PKT" }, { "name": "arch", "value": "x86_64,x86" }, { "name": "name", "value": "Awesome OS" } ], "dependentProductIds": [], "href": "/products/00", "id": "00", "multiplier": 1, "name": "Awesome OS", "productContent": [ { "content": { "arches": null, "contentUrl": "/content/6/$releasever/$basearch/debug", "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-awesome-os", "id": "FFFF", "label": "awesome-os-debug-rpms", "metadataExpire": 86400, "modifiedProductIds": [ "0A" ], "name": "Awesome OS (Debug RPMs)", "releaseVer": null, "requiredTags": "awesome-os-server", "type": "yum", "vendor": "Candlepin" }, "enabled": false } ] }

You get a bunch of files that look like this:

{
    "attributes": [
        {
            "name": "type",
            "value": "PKT"
        },
        {
            "name": "arch",
            "value": "x86_64,x86"
        },
        {
            "name": "name",
            "value": "Awesome OS"
        }
    ],
    "dependentProductIds": [],
    "href": "/products/00",
    "id": "00",
    "multiplier": 1,
    "name": "Awesome OS",
    "productContent": [
        {
            "content": {
                "arches": null,
                "contentUrl": "/content/6/$releasever/$basearch/debug",
                "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-awesome-os",
                "id": "FFFF",
                "label": "awesome-os-debug-rpms",
                "metadataExpire": 86400,
                "modifiedProductIds": [
                    "0A"
                ],
                "name": "Awesome OS (Debug RPMs)",
                "releaseVer": null,
                "requiredTags": "awesome-os-server",
                "type": "yum",
                "vendor": "Candlepin"
            },
            "enabled": false
        }
    ]
}

This morning I came in to find that my root partition on my workstation was full, eaten up by the development VM (virtual machine) that had grown to 145GB. SIGH! This has happened in the past and usually I delete old VMs, move VMs to my home directory on /home etc.

Today I decided it was time to fix the problem. The root partition is 193G and /home is 280G. The home dir has about 210G free and after removing the development VM from root, it now has 150GB free. If I merge the two partitions I’ll have 360GB of free space to use for VMs etc. I will lose the nice ability to reinstall linux and not format /home partition as now it will be part of root but it avoids the day to day minutiae of managing free space between the two partitions.

I’m fortunate that I can free up the space on root enough to copy the contents of /home over to it. Otherwise the process would involve freeing up space on /home and reusing it on /root, blah blah blah.

First let’s copy /home to a new directory on root, this will create a /home-new/home but that’s ok.

rsync -avzX /home /home-new

Next, let’s unmount and remove /home.

umount /home
lvremove /dev/vg0/home

That freed up 280G, so we can now grow the root partition to use up that space.

lvextend -l +100%FREE /dev/vg0/root

So far so good, now we need to grow the filesystem of the root partition to use the new space as well. I’m using XFS, make sure you use the proper filesystem tool to resize the one you are using.

xfs_growfs /

Let’s restore home.

mv /home-new/home/* /home/

Update /etc/fstab not to mount /home as a different partition. Then reboot just to make sure things are good.

Go has no class

Coming from Java, Python, and Ruby, I’m used to working with objects and methods. Go doesn’t have classes but it does have structs that you can add methods to it.

In Java, you would typically do the following:

public class User {
  private String name;

  public User(String name) {
    this.name = name;
  }

  // method with return value
  public String getName() {
    return this.name;
  }

  // method with argument
  public void setName(String name) {
    this.name = name;
  }

  // method with argument and return value
  public int argWithReturn(String anarg) {
    return anarg.length();
  }
}

In Go you would use a struct.

type User struct {
    name string
}

The weirdest part is that you don’t define the methods in the struct block. How do you add methods to the User struct? Simply create a function and add (u *User) to it.

func (u *User) GetName() string {
    return u.name
}

func (u *User) SetName(nm string) {
    u.name = nm
}

func (u *User) ArgWithReturn(anarg string) int {
    return len(anarg)
}

Whoa, that’s strange. Why does GetName return a *User type? It doesn’t, the return type is defined at the end of the definition. So GetName actually returns a string. The (u *User) is how you tell Go these methods are attached to the User struct.

I think I’ll look either into packages or argument parsing.

Adventures with the Go programming language

After 10+ years of working with Java, I spent the last year working with Ruby & Rails. Let’s just say I’m not a fan. I wanted to learn something new and I’ve heard a lot of good things about Go. Quite a few projects use Go: Docker, Hugo, Kubernetes, Prometheus, and many more.

Three weeks ago, I picked up a copy of “The Go Programming Language” by Donovan & Kernighan. Yes, the same Kernighan that co-wrote the famous, “The C Programming Language”.

In order to learn it, I needed a real life scenario, I needed a project. Why not port, sm_photo_tool, my Smugmug command line tool from python to Go lang? That will give me experience with how Go handles parsing arguments, classes, REST APIs, concurrent uploads, and a bunch more things.  It’s the perfect project to learn Go.

In the next post I’ll talk about structs in Go.