Menu

Home

Drawing sketch: Blue Hair

kinow @ May 16, 2016 11:13:03
Blue Hair
Blue Hair

For redditgetsdrawn

Some Linux commands I used this week

kinow @ May 06, 2016 23:26:03

These are some commands I used on Linux servers this week. Adding them here in case someone else find them interesting, and also due to my bad memory :-)

Listing latest installed packages in SLES

rpm -qa --last

This will display the last packages installed. Useful when there are packages being updated, and you need to confirm what changed, and when.

Listing packages in SLES and origin repository

rpm -qa --qf '%-30{DISTRIBUTION} %{NAME}\n'| sort

The output will have two columns. The first containing the repository name, and the second column with the package name. For example.

devel:languages:R:base / SLE_11_SP2 R-base
devel:languages:R:base / SLE_11_SP2 R-base-devel
home:flacco:sles / SLE_11_SP3 php53-phar
home:happenpappen / SLE_11_SP2 nodejs

Grep for content in XML tags

Be it for web services, or for finding things in Jenkins XML files. Being able to grep the tag attribute or tag name might be useful. Look at the following example that uses the books XML provided by Microsoft for testing.

grep -oP "(?<=<genre>).*?(?=</genre>)" books.xml | sort | uniq

Which will outputs the following.

Computer
Fantasy
Horror
Romance
Science Fiction

Find Python site packages directory

Sometimes you have Anaconda, but also the system installation, and maybe even other Python distributions. Knowing where Python is looking for site packages can be helpful to confirm the package exists, and also to inspect its sources.

python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"

An example of the output of the script.

/usr/lib/python2.7/dist-packages

Force no-cache via curl for a list of files

Useful when you have a proxy like squid caching some requests from an application and you want to flush the cache and get the latest content (which will be cached again, but then you can fix it once confirmed).

curl --silent -H 'Cache-Control: no-cache' http://systemcachingvalues.local/somedoc.html

Find to which servers a Linux process is talking to

You have to find the pid of the process that you would like to investigate (e.g. 6364) and have strace installed.

strace -p 6364 -f -e trace=network -o output.txt

The command above creates output.txt with the trace information. Then you can grep for the IP addresses with the following regex.

grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)" output.txt

Which will output something similar to the following example.

127.0.1.1
127.0.1.1
127.0.1.1
192.168.20.4
10.10.0.12
...

And finally, you can call dig to get the server name, and also remove duplicates.

grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)" output.txt | xargs -l dig +noall +answer +nocmd -x | awk '{ print $5}' | sort | uniq

Which gives you the following.

ec2-52-13-43-205.compute-1.amazonaws.com.
ec2-52-32-244-147.compute-1.amazonaws.com.
ec2-52-44-11-85.compute-1.amazonaws.com.
ec2-52-55-36-20.us-west-2.compute.amazonaws.com.
ec2-52-11-19-24.us-west-2.compute.amazonaws.com.
ec2-52-2-21-13.compute-1.amazonaws.com.
ec2-54-33-249-49.us-west-2.compute.amazonaws.com.
ec2-54-180-165-17.us-west-2.compute.amazonaws.com.
ec2-54-2-177-91.compute-1.amazonaws.com.
ec2-54-8-163-15.compute-1.amazonaws.com.
syd11s01-in-f124.1e110.net.
syd11s02-in-f5.1e110.net.
syd12s02-in-f3.1e110.net.
...

That’s all for today.

Happy hacking!

Using Active Choices with Role Strategy Plug-in

kinow @ Apr 24, 2016 16:05:03

Having worked in Open Source for a few years, one of my favorite things is when you can share experience with other people that you meet. Andrew Gray has worked with .NET and Jenkins for years, and we met through Open Source. He has helped me in the past with Jenkins and .NET, and also maintains the blog Jenkins.NET.

A couple of days ago he sent me an interesting question. He asked me if that would be possible to use Active Choices Plug-in with the Role Strategy Plug-in. This plug-in lets you define roles, define which permissions a role has, and then assign users to the roles.

The idea was to show a set of parameters that would vary according to the user roles. So, say that the user is a tester. Then s/he would see certain parameters. But if s/he was a manager, other parameters would be displayed.

To the code

import hudson.model.User
import hudson.model.Hudson
import hudson.security.AuthorizationStrategy
import hudson.security.Permission
import com.michelin.cio.hudson.plugins.rolestrategy.RoleBasedAuthorizationStrategy
import com.michelin.cio.hudson.plugins.rolestrategy.RoleMap

AuthorizationStrategy strategy = Hudson.getInstance().getAuthorizationStrategy();

jobs = []
user = User.current()
userId = user.getId()

if (strategy != null 
    // not very straightforward to get the groups for a given user
    roles = roleStrategy.getGrantedRoles("globalRoles")
    for (entry in roles) {
        role = entry.key
        users = entry.value
        if (role.getName().equals("tester")) {
            if (userId in users) {
                jobs = ["PROJECT_FOR_TESTERS1", "PROJECT_FOR_TESTERS2"]
                break
            }
        } else if (role.getName().equals("admin")) {
            if (userId in users) {
                jobs = ["PROJECT_FOR_ADMINS1", "PROJECT_FOR_ADMINS2"]
                break
            }
        }
    }
}

return jobs
// TODO: handle anonymous user ;-)

And now let’s dissect the code.

import hudson.model.User
import hudson.model.Hudson
import hudson.security.AuthorizationStrategy
import hudson.security.Permission
import com.michelin.cio.hudson.plugins.rolestrategy.RoleBasedAuthorizationStrategy
import com.michelin.cio.hudson.plugins.rolestrategy.RoleMap

You start by importing the classes that you need.

AuthorizationStrategy strategy = Hudson.getInstance().getAuthorizationStrategy();

jobs = []
user = User.current()
userId = user.getId()

The first line gets the current AuthorizationStrategy used in Jenkins.

Then we create an empty array of jobs, which is the value returned by default. And get the current logged in user ID.

if (strategy != null 
    // not very straightforward to get the groups for a given user
    roles = roleStrategy.getGrantedRoles("globalRoles")
    for (entry in roles) {
        role = entry.key
        users = entry.value
        if (role.getName().equals("tester")) {
            if (userId in users) {
                jobs = ["PROJECT_FOR_TESTERS1", "PROJECT_FOR_TESTERS2"]
                break
            }
        } else if (role.getName().equals("admin")) {
            if (userId in users) {
                jobs = ["PROJECT_FOR_ADMINS1", "PROJECT_FOR_ADMINS2"]
                break
            }
        }
    }
}

return jobs
// TODO: handle anonymous user ;-)

The final part simply iterates through each existing role, and then through the users in that role. I could not find a more ellegant way of doing that, but in case you would like to maybe optimize the code, here are the main classes that I used from Jenkins to write the script.

Normally I either use my local working copy of Jenkins project imported in Eclipse, or use the “Find File” feature in GitHub. Then remember a little bit about Jenkins Java API to walk through the classes and write the Groovy code (I made a video about writing Groovy with Auto Complete in Eclipse, but that’s in Portuguese only).

What the result looks like in Jenkins

In this section we have just simple screenshots, showing the script in the job configuration.

Screenshot 01
Screenshot 01

And the resulting screen. The parameter DeployAPP will be available during the build, and can be used to trigger other jobs or pipelines.

Screenshot 02
Screenshot 02

Where can you find this script?

This blog post has a copy of the script, but I have also submitted a pull request to a repository maintained by another friend, Ioannis Moutsatsos. Ioannis maintains a Jenkins installation in Novartis, and is probably the most skillful user of the Active Choices Plug-in.

Another advantage of this other copy, is that it may be updated with time, in case there are bugs or improvements. So watch Ioannis’ repository for updates!

ps: I’ve also met Ioannis through Open Source code, and have learned a lot by working with him.

♥ Open Source!

Drawing vector art: Kumamoto Kenjinkai mascot

kinow @ Apr 19, 2016 21:39:03
Kumamoto Kenjinkai mascot
Kumamoto Kenjinkai mascot

For Brazilian Kumamoto Kenjinkai. Done with Macromedia Freehand MX, some long time ago.

Trying SaltStack with Docker

kinow @ Apr 17, 2016 22:06:03

Some weeks ago I started learning SaltStack for a project at work. But I couldn’t find a good Docker image for that and I had to ask the Ops team for some VM’s. We are having a rainy weekend in Auckland, so I decided to have another look at the Jenkins SaltStack Plug-in.

But now since I was at home, I couldn’t use the VM’s that I had access to at work. So decided to look again at Docker or Vagrant images. After playing with a few images, I found bbinet/salt-master. It not only sets up a master, but also provides an easy way to enable the cherrypy API (necessary for the Jenkins plug-in).

This post describes the steps that I took to have a running Salt Master with the API enabled. First you need to create some directories and files to use with the image.

shell$ mkdir ~/master && cd ~/master
shell$ mkdir -p config/master.d/
shell$ vim config/master.d/api.conf

The api.conf contains the SaltStack API configuration. You can change port, user and other settings if necessary. Just remember to add a credential in Jenkins for the plug-in.

# File: api.conf
external_auth:
  pam:
    saltapiuser:
      - .*
      - '@runner'
      - '@wheel'
      - '@jobs'
rest_cherrypy:
  port: 8000
  host: 0.0.0.0
  disable_ssl: True
  static: /opt/molten
  static_path: /assets
  app: /opt/molten/index.html
  app_path: /molten

The image also conveniently provides a script that is executed before the entry point (if provided). So we can also create a user for the API automatically when the image is created.

shell$ vim config/before-exec.sh
#!/bin/bash
# File: before-exec.sh
useradd saltapiuser
echo -e "nosecret\nnosecret\n" | passwd saltapiuser
exit 0

Also make the script executable.

chmod +x config/before-exec.sh

And finally start the container.

docker run --name salt-master -v $PWD/config:/config \
    -p 4505:4505 -p 4506:4506 -p 443:443 -p 8000:8000 \
    bbinet/salt-master

Once the container is running, you can go to http://localhost:8000 and log in as saltapiuser:nosecret, and also configure your plug-in in Jenkins.

Happy hacking!