vij update

Vij is a bash function that makes editing specific files easier to find in a deeply nested directory structure. At the time I was working in Java which is notorious for having a deep directory structure. In 2008, I updated vij to be more powerful by offering a list of files matching the name, giving you the option to make a choice.

Our current project is in Golang and I’ve found a deficiency in vij. We have a vendor directory which contains our dependencies and sometimes vij will find files in the vendor directory which I don’t want to edit. It would be nice to exclude a directory from the find command used by vij.

I always want to ignore the vendor directory. But I also want to allow arbitrary names to also be filtered out. The existing vij implementation results in the following crazy output.

$ vij util.go

Multiple matches found...
1: ./pkg/broker/util.go
2: ./pkg/fusortest/util.go
3: ./vendor/k8s.io/kubernetes/vendor/golang.org/x/crypto/ssh/terminal/util.go
4: ./vendor/k8s.io/kubernetes/vendor/github.com/vmware/photon-controller-go-sdk/photon/util.go
5: ./vendor/k8s.io/kubernetes/vendor/github.com/spf13/viper/util.go
6: ./vendor/k8s.io/kubernetes/vendor/github.com/spf13/cobra/doc/util.go
...
16: ./vendor/k8s.io/kubernetes/vendor/github.com/kr/pty/util.go
...
108: ./vendor/github.com/coreos/etcd/client/util.go
q: Quit

? 

I just want to see the util.go files in my code tree and ignore all of the ones in vendor. I changed the dafiles variable to have two more options. One is always ignore vendor from the normal use case, but also allow the user to pass in a string to filter on.

    if [ "$3" == "" ]; then
        dafiles=$(find . -type f -name "$2" -not -path "*/vendor/*")
    else 
        dafiles=$(find . -type f -name "$2" | grep -v $3)
    fi

Now when I run vij util.go, vij ignores the vendor directory.

$ vij util.go

Multiple matches found...
1: ./pkg/broker/util.go
2: ./pkg/fusortest/util.go
q: Quit

? 

Let’s say I want to find all files named User*.java but not the ones in the test directory. You can simply supply “test” as the second argument to vij.

BEFORE

$ vij User*.java

Multiple matches found...
1: ./src/main/java/org/candlepin/auth/permissions/UserUserPermission.java
2: ./src/main/java/org/candlepin/auth/permissions/UsernameConsumersPermission.java
3: ./src/main/java/org/candlepin/auth/UserAuth.java
4: ./src/main/java/org/candlepin/auth/UserPrincipal.java
5: ./src/main/java/org/candlepin/model/User.java
6: ./src/main/java/org/candlepin/model/UserCurator.java
7: ./src/main/java/org/candlepin/resource/UserResource.java
8: ./src/main/java/org/candlepin/service/UserServiceAdapter.java
9: ./src/test/java/org/candlepin/auth/UserPrincipalTest.java
10: ./src/test/java/org/candlepin/auth/permissions/UsernameConsumersPermissionTest.java
11: ./src/test/java/org/candlepin/model/UserTest.java
12: ./src/test/java/org/candlepin/resource/UserResourceTest.java
q: Quit

? 

AFTER

$ vij User*.java test

Multiple matches found...
1: ./src/main/java/org/candlepin/auth/permissions/UserUserPermission.java
2: ./src/main/java/org/candlepin/auth/permissions/UsernameConsumersPermission.java
3: ./src/main/java/org/candlepin/auth/UserAuth.java
4: ./src/main/java/org/candlepin/auth/UserPrincipal.java
5: ./src/main/java/org/candlepin/model/User.java
6: ./src/main/java/org/candlepin/model/UserCurator.java
7: ./src/main/java/org/candlepin/resource/UserResource.java
8: ./src/main/java/org/candlepin/service/UserServiceAdapter.java
q: Quit

? 

Here’s the full implementation of the updated vij:

# reused by other commands that require a filename
__dafiles ()
{
    cmd=$1
    if [ "$3" == "" ]; then
        dafiles=$(find . -type f -name "$2")
        #dafiles=$(find . -type f -name "$2" -not -path "*/vendor/*")
    else
        dafiles=$(find . -type f -name "$2" | grep -v $3)
    fi
    matches=$(echo $dafiles | gawk '{print NF}')

    case "$matches" in
        0)
           echo "No matches found"
           show=""
           ;;
        1)
           show=$dafiles
           ;;
        *)
           echo
           echo "Multiple matches found..."
           i=1
           for option in $dafiles
           do
              echo "$i: $option"
              i=`expr $i + 1`
           done
           echo "q: Quit"
           echo 
           read -p "? " ans
           if [ "q" == "$ans" ]; then
              show=""
           else
              show=$(echo $dafiles | gawk '{print $'$ans'}')
           fi
           ;;
    esac

    if [ "" != "$show" ]; then
       $cmd $show
    fi
}

vij ()
{
    __dafiles "vim" $1 $2
}

Private Golang, reporting for duty

Many modern programming languages have a mechanism for controlling the visibility of members and methods. Most of the languages use keywords like private and public.

Java and C++ use public, private, and protected. Ruby also uses keywords: private, and protected.

Python on the other hand uses convention to control visibility, prefixing your methods with _ or __. Even then it’s more of a suggestion in Python as you can call it anyway. There is nothing to prevent it from actually being called from other code. It’s an understanding among Python developers that if there is an underscore you shouldn’t call it.

Go also has no keyword to control visibility. What no keyword? There has to be something? Maybe it uses _ like python? Nope that won’t do anything. Then what would Go use to control visibility? Go uses a very simple syntax, capitalization.

CAPITALIZATION? What? Yes, that’s correct. A simple capital letter makes the member or method public. Lowercase member, methods are private.

// GetName is a public method of User struct
func (u *User) GetName() string {
  return u.name
}

// nickName is a private method of User struct
func (u *User) nickName() {
  // do something
}

The same logic applies to members. Given the following example struct, Foo is public and bar is private:

type Example struct {
  Foo string
  bar string
}

So if you want to make your Go methods or members private, just start them with a lowercase letter. Want others to use it, capitalize it.

Go has no class

Coming from Java, Python, and Ruby, I’m used to working with objects and methods. Go doesn’t have classes but it does have structs that you can add methods to it.

In Java, you would typically do the following:

public class User {
  private String name;

  public User(String name) {
    this.name = name;
  }

  // method with return value
  public String getName() {
    return this.name;
  }

  // method with argument
  public void setName(String name) {
    this.name = name;
  }

  // method with argument and return value
  public int argWithReturn(String anarg) {
    return anarg.length();
  }
}

In Go you would use a struct.

type User struct {
    name string
}

The weirdest part is that you don’t define the methods in the struct block. How do you add methods to the User struct? Simply create a function and add (u *User) to it.

func (u *User) GetName() string {
    return u.name
}

func (u *User) SetName(nm string) {
    u.name = nm
}

func (u *User) ArgWithReturn(anarg string) int {
    return len(anarg)
}

Whoa, that’s strange. Why does GetName return a *User type? It doesn’t, the return type is defined at the end of the definition. So GetName actually returns a string. The (u *User) is how you tell Go these methods are attached to the User struct.

I think I’ll look either into packages or argument parsing.

Adventures with the Go programming language

After 10+ years of working with Java, I spent the last year working with Ruby & Rails. Let’s just say I’m not a fan. I wanted to learn something new and I’ve heard a lot of good things about Go. Quite a few projects use Go: Docker, Hugo, Kubernetes, Prometheus, and many more.

Three weeks ago, I picked up a copy of “The Go Programming Language” by Donovan & Kernighan. Yes, the same Kernighan that co-wrote the famous, “The C Programming Language”.

In order to learn it, I needed a real life scenario, I needed a project. Why not port, sm_photo_tool, my Smugmug command line tool from python to Go lang? That will give me experience with how Go handles parsing arguments, classes, REST APIs, concurrent uploads, and a bunch more things.  It’s the perfect project to learn Go.

In the next post I’ll talk about structs in Go.

SSL sucks

I swear every time I start going down the SSL path I get similar errors and never remember what they mean. Trying to get my app talking to qpid broker using the JMS client.


Caused by: javax.net.ssl.SSLException: Received fatal alert: bad_certificate
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[na:1.7.0_55]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1630) ~[na:1.7.0_55]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1598) ~[na:1.7.0_55]
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1767) ~[na:1.7.0_55]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1063) ~[na:1.7.0_55]
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:887) ~[na:1.7.0_55]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:761) ~[na:1.7.0_55]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[na:1.7.0_55]
at org.apache.qpid.transport.network.security.ssl.SSLReceiver.received(SSLReceiver.java:102) ~[qpid-common-0.22.jar:na]
... 3 common frames omitted

Eclipse you suck!

I’ve spent several hours trying to get the Android Developer Toolkit version of eclipse to run on Fedora 20. It’s constantly keeps dying with a segfault and libsoup.

[13899.331278] java[15520]: segfault at 0 ip 0000003929c70061 sp 00007fb66bd4df40 error 4 in libsoup-2.4.so.1.7.0[3929c00000+a8000]

I’ve tried all the workarounds on the net which suggest to make it use mozilla, so far no luck.

-Dorg.eclipse.swt.browser.DefaultType=mozilla
-Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib64/xulrunner/

I even deleted my $HOME/.eclipse, that didn’t work. I switched to a new workspace specifically for ADT. Again didn’t help, still constantly dies.

UGH! Piece of Garbage. And to Ajay, no IntelliJ comments 🙂

Candlepin 0.5.5 released

Candlepin

It’s that time again, another release of Candlepin and associated projects available for your enjoyment. With this release we have subscription-manager in Fedora as well as a debut build of Thumbslug.

For more information on Candlepin, please visit: http://candlepinproject.org/

Features & Enhancements

subscription-manager

  • added support for host registration and guest association when host can not register itself
  • virt-who work to handle ESX guests
  • subscription-manager available in fedora

candlepin

  • build modified to use the tito hotness instead of bunch of disjoint bash scripts
  • disable manifest rules import
  • added support for host registration and guest association when host can not register itself

thumbslug

  • added appropriate init scripts to run as a service
  • uses Candlepin CRL
  • thumbslug talks to akamai
  • created puppet module for katello

Bugs fixed

subscription-manager

705883 Fix error dialog modal issues.
719743 Improved text output for successful pool subscription
740788 Getting error with quantity subscribe using subscription-assistance page.
746259 Don’t allow the user to pass in an empty string as an activation key
746732 Only use fallback locales for dates we need to parse
749332 Normalize the error messages for not being registered
749636 Client should not support users entering activation keys and existing consumer ids
752572 add interval logging statements back in on rhsmcertd startup
753093 The available subscriptions count does not show correctly in Subscription Manager GUI
754821 Default org of “Unknown” was not marked for gettext
755031 Unregister before attempting to run a second registration
755035 Migration script should work on RHEL 5.7 and up.
755130 add extra whitespace to classic warning
755541 Enhanced the message in the katello plugin to debug when the backend system does not support environments.
756173 Unexpected behavior change in subscription-manager unregister
756507 do not use output from “getlocale” as input for “setlocale”
758471 install-num-migrate-to-rhsm threw traceback when no instnum was found.
759199 rhsmcertd is logging the wrong value for certFrequency

candlepin

753093 The Available Subscriptions count do not show correctly in Subscription Manager GUI
754841 Implement DELETE /pools/id.
754843 Fix legacy virt bonus pools missing pool_derived.
755677 Activation Keys should not check quantity on unlimited pools
756628 Translate missing rule errors.
758462 ensure job detail isn’t null, skip it.

thumbslug

759607 update url for subscriptions handler

Download & Setup

Make sure you read over the Candlepin Setup Guide, which is located at https://fedorahosted.org/candlepin/wiki/Setup.

As well as the Headpin Install Guide which can be found at https://fedorahosted.org/candlepin/wiki/headpin/Install

Just give me the bits already! You can get the various bits at the urls below.

Candlepin:
http://repos.fedorapeople.org/repos/candlepin/candlepin/

Thumbslug:
http://repos.fedorapeople.org/repos/candlepin/thumbslug/

Headpin:
http://repos.fedorapeople.org/repos/katello

Subscription Manager:
http://repos.fedorapeople.org/repos/candlepin/subscription-manager/