Pretty Print directory of .json files

I had a bunch of compressed json files that I needed to pretty print to make them more readable. This little snippet will create a new pretty printed json file prefixed with pp:

ls *.json | xargs -I {} sh -c "cat {} | python -mjson.tool > pp{}"

Instead of having to look at files that look like this:

{ "attributes": [ { "name": "type", "value": "PKT" }, { "name": "arch", "value": "x86_64,x86" }, { "name": "name", "value": "Awesome OS" } ], "dependentProductIds": [], "href": "/products/00", "id": "00", "multiplier": 1, "name": "Awesome OS", "productContent": [ { "content": { "arches": null, "contentUrl": "/content/6/$releasever/$basearch/debug", "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-awesome-os", "id": "FFFF", "label": "awesome-os-debug-rpms", "metadataExpire": 86400, "modifiedProductIds": [ "0A" ], "name": "Awesome OS (Debug RPMs)", "releaseVer": null, "requiredTags": "awesome-os-server", "type": "yum", "vendor": "Candlepin" }, "enabled": false } ] }

You get a bunch of files that look like this:

{
    "attributes": [
        {
            "name": "type",
            "value": "PKT"
        },
        {
            "name": "arch",
            "value": "x86_64,x86"
        },
        {
            "name": "name",
            "value": "Awesome OS"
        }
    ],
    "dependentProductIds": [],
    "href": "/products/00",
    "id": "00",
    "multiplier": 1,
    "name": "Awesome OS",
    "productContent": [
        {
            "content": {
                "arches": null,
                "contentUrl": "/content/6/$releasever/$basearch/debug",
                "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-awesome-os",
                "id": "FFFF",
                "label": "awesome-os-debug-rpms",
                "metadataExpire": 86400,
                "modifiedProductIds": [
                    "0A"
                ],
                "name": "Awesome OS (Debug RPMs)",
                "releaseVer": null,
                "requiredTags": "awesome-os-server",
                "type": "yum",
                "vendor": "Candlepin"
            },
            "enabled": false
        }
    ]
}

This morning I came in to find that my root partition on my workstation was full, eaten up by the development VM (virtual machine) that had grown to 145GB. SIGH! This has happened in the past and usually I delete old VMs, move VMs to my home directory on /home etc.

Today I decided it was time to fix the problem. The root partition is 193G and /home is 280G. The home dir has about 210G free and after removing the development VM from root, it now has 150GB free. If I merge the two partitions I’ll have 360GB of free space to use for VMs etc. I will lose the nice ability to reinstall linux and not format /home partition as now it will be part of root but it avoids the day to day minutiae of managing free space between the two partitions.

I’m fortunate that I can free up the space on root enough to copy the contents of /home over to it. Otherwise the process would involve freeing up space on /home and reusing it on /root, blah blah blah.

First let’s copy /home to a new directory on root, this will create a /home-new/home but that’s ok.

rsync -avzX /home /home-new

Next, let’s unmount and remove /home.

umount /home
lvremove /dev/vg0/home

That freed up 280G, so we can now grow the root partition to use up that space.

lvextend -l +100%FREE /dev/vg0/root

So far so good, now we need to grow the filesystem of the root partition to use the new space as well. I’m using XFS, make sure you use the proper filesystem tool to resize the one you are using.

xfs_growfs /

Let’s restore home.

mv /home-new/home/* /home/

Update /etc/fstab not to mount /home as a different partition. Then reboot just to make sure things are good.

Go has no class

Coming from Java, Python, and Ruby, I’m used to working with objects and methods. Go doesn’t have classes but it does have structs that you can add methods to it.

In Java, you would typically do the following:

public class User {
  private String name;

  public User(String name) {
    this.name = name;
  }

  // method with return value
  public String getName() {
    return this.name;
  }

  // method with argument
  public void setName(String name) {
    this.name = name;
  }

  // method with argument and return value
  public int argWithReturn(String anarg) {
    return anarg.length();
  }
}

In Go you would use a struct.

type User struct {
    name string
}

The weirdest part is that you don’t define the methods in the struct block. How do you add methods to the User struct? Simply create a function and add (u *User) to it.

func (u *User) GetName() string {
    return u.name
}

func (u *User) SetName(nm string) {
    u.name = nm
}

func (u *User) ArgWithReturn(anarg string) int {
    return len(anarg)
}

Whoa, that’s strange. Why does GetName return a *User type? It doesn’t, the return type is defined at the end of the definition. So GetName actually returns a string. The (u *User) is how you tell Go these methods are attached to the User struct.

I think I’ll look either into packages or argument parsing.

Adventures with the Go programming language

After 10+ years of working with Java, I spent the last year working with Ruby & Rails. Let’s just say I’m not a fan. I wanted to learn something new and I’ve heard a lot of good things about Go. Quite a few projects use Go: Docker, Hugo, Kubernetes, Prometheus, and many more.

Three weeks ago, I picked up a copy of “The Go Programming Language” by Donovan & Kernighan. Yes, the same Kernighan that co-wrote the famous, “The C Programming Language”.

In order to learn it, I needed a real life scenario, I needed a project. Why not port, sm_photo_tool, my Smugmug command line tool from python to Go lang? That will give me experience with how Go handles parsing arguments, classes, REST APIs, concurrent uploads, and a bunch more things.  It’s the perfect project to learn Go.

In the next post I’ll talk about structs in Go.

Cold but fun day

Today was a cold but fun day. We started the day with Iliana’s 8:30am softball game (they won 8-5). Adan had a pitching lesson at 10:45am where he has improved his fastball and change up pitches. He worked on his curve and some pickoff move all while battling cold, wind and what felt like a sandstorm from the field. Home for about 2 hours to eat lunch and warm up before heading out again, this time for Marco’s 2:30 soccer game. The won that game 5-1 with some outstanding ball movement. Still frozen, we weren’t done yet. Off to Marco’s baseball game. His team won that game, but the best part was Marco hit a SOLO HOME RUN over the centerfield fence. It was AWESOME! We finished the evening with dinner at Chipotle and watched Star Wars the Force Awakens when we got home.

Installed rear shocks on Mazdaspeed

We had a rare free day this weekend, usually they’re filled up with softball, baseball and/or soccer games. I took the free day to do some car maintenance. I’m in desperate need of new shocks and struts, already ate through one set of tires. Saturday I replaced the rear shocks. It took me 4 hours, taking pictures and reading instructions that came with the parts.

It’s pretty difficult finding parts for the Mazdaspeed version of the Mazda 3, it’s either OEM parts from the Mazda dealer which is clearly $$$, or it’s some performance parts. I was trying to balance price with performance, so I ended up with the Corksport adjustable shocks and struts. They are the cheapest yet still offer some level of performance. I really wanted to get the spring combo but that was another $200.

 

Why not take it to a shop? Well the last 2 quotes I got were about $250+ in labor for the rear shocks. That’s too rich for my blood especially for removing 4 bolts. Here’s a high level description of what I had to do, see the Corksport site for detailed instructions and be sure to checkout the Mazdaspeed forums for advice as well.

  1. Raise vehicle onto jack stands, since you’ll need the jack for a later step IMG_0691
  2. Use the floor jack on the lower control arm to compress the rear spring, and to alleviate pressure on the shocks.IMG_0701
  3. Use a 17mm socket to remove the lower bolt of shock  IMG_0693
  4. Use a 12mm deep socket to remove the two nuts holding the top mount in place.IMG_0697
  5. Now you need to manually compress the shock to allow it to clear the lower bracket in order to be removed from the vehicle.
  6. Once removed, used a 13mm socket to remove the nut holding the aluminum mount to the shock. I had to use a pair of vise grips to hold the shock bolt from moving while removing the nut. The stock bump stops were shot.IMG_0704

That is the standard removal steps. Now the rest are Corksport specific. For the Corksport shocks you only need to keep the aluminum mount.

  1. Retain the stock aluminum mounts, remove the stock bump stops (they’re probably trash anyway), and the stock dust cover.
  2. Place the aluminum mount onto the Corksport shock. You will notice there isn’t much bolt that comes through the mount. This took me a while to figure out. Press the mount down onto the shock, you might have to compress the shock, then put the nut on the bolt and start turning until it latches. This took a bit of pressure and patience. Tighten the nut to 18 lb-ft of torque.
  3. The bump stops pre-installed on the Corksport shocks will compress to allow you to properly torque down the nut and install the mount.

At this point the old shocks will have been removed, the new ones have the mounts installed, and you are ready to install the new ones on the vechicle.

  1. Manually compress the shocks so you can get the shocks into the wheel well.
  2. Insert the top mount onto the 2 mounting bolts.
  3. Align the bottom to the lower bracket. You have 2 options here.
    1. Let the bottom go through the bracket to be compressed by hand later
    2. Keep the shock compressed while you insert the 17mm bolt to hold the bottom of the shock in place.IMG_0702
  4. Install the 12mm nuts on the top mounting bolts. Torque them down to 18 lb-ft.
  5. Now install the 17mm bolt on the bottom and torque it down to 50 lb-ft.
  6. Lower the floor jack to relieve the pressure on the spring.
  7. Re-install the tire and you’re DONE! IMG_0699 IMG_0698
  8. Repeat for the other side.

Now I need to replace the front struts. Those are not as trivial since they involve compressing the springs on the strut, and lot’s of penetrating oil. We’ll see if $250+ is worth it for the front or if I feel inspired to do this myself.

Happy 14th Birthday Adan!

Today we celebrate Adan’s 14th birthday. He’s grown a lot both physically and mentally🙂 He’s becoming quite a wonderful young man.

Here he is having fun with some sunglasses I got from work.

21292971852_ab3a3e0e9a_k

His baseball jersey.

21803625642_430849b916_k

Soccer player

Marco’s been super busy playing soccer with both his middle school team and his club soccer team. He’s not getting a lot of playing time on middle school during the conference games which he expected being one of the few 7th graders on a team full of 8th grade veterans. But his club soccer team hasn’t been playing him as often as he wanted despite working equally hard.

So to hopefully cheer him up and keep up his spirits I write him a poem, I guess you can call it a poem. At 5:40am I came up with this (I revised it a bit to add more).

Soccer Player by Jesus Rodriguez

When I’m on the bench, I do not feel fine.
I’m not a bench warmer, I’m putting in time.

When I play defense, I will be agile and fast.
When I play defense, nothing will get past.

When I play offense, I will not bore.
As the offense, I will shoot and score.

As the goalie, I will block all.
Because as the goalie, I am a brick wall.

Nothing will cause dismay.
Because soccer is what I play.

Fun with cli file transformation

I used the Wake County property tax page to get a list of all of the homes in our neighborhood. I pasted that into a spreadsheet. The Proper and Proper last columns are actually cells using the =PROPER() function. It takes the given cell and converts it from all upper case to proper case. The resulting list looks like this:

Address Street Last name Owners Proper Proper last E-mail
123 SCRABBLES DR JONES FRED Fred Jones neighbor1@email.com
456 SCRABBLES DR BONES WILMA Wilma Bones neighbor2@eahoo.com

I probably could’ve done this whole thing without using the spreadsheet or the =PROPER() function but it was easier considering the original data came from a table. I then exported that as a CSV file.

Address,Street,Last name,Owners,Proper,Proper last,E-mail,
123,SCRABBLES DR,JONES,FRED,Fred,Jones,neighbor1@email.com,
456,SCRABBLES DR,BONES,WILMA,Wilma,Bones,neighbor2@eahoo.com,

What I’m trying to do is get a list with the following format:

"First Lastname" <emailaddress@domain.com>, "First Lastname" <email2@domain.com>

I did it with the following command line. The trickiest was getting awk to print out the double quotes uses “\42”

cat neighbors.csv | grep -v ^Address | awk -F , '{print "\42"$5" "$6"\42 <"$7">,"}' | grep -v "<>" | tr '\n' ' ' > full-list.txt

I then copy and pasted the resulting file contents into the “Direct add members” feature. The one tidbit that I had to do was only copy and paste 10 addresses at once. I probably could’ve updated the above script to do that for me. But by this point I got 90% of what I needed so I just split the rest by hand.