Jira Administration - Custom Fields & Screens

Jira has a complex administration concepts. Performing Jira admin tasks without understanding the complete flow leads to mistakes. In this post, I'm explaining how the Custom Fields are created and how to link it to project screens.

Creating custom fields
Let's assume you get a request to create a custom field "Severity". Check whether this custom field is already existing by

• Login to Jira -> Administration -> Issue Fields -> Custom Fields

If it is not available, create it straight away

• Login to Jira -> Administration -> Issue Fields -> Custom Fields -> Add custom Field -> Choose the field type -> Provide field name, search template, etc , applicable issue types, Applicable context.

   Custom Field Context

        The custom field which we created can be associated to all issues by selecting "Global context" or to specific projects. Depending on the usability of the custom field make it either global or project specific.

If the custom field is already exist, then try to reuse it.

Screens

Screens group multiple issue fields. Using Screens, you can control which fields are displayed. You can also split fields on a Screen into multiple tabs.

Issue: Nexus is too slow in downloading pom files

Recently our nexus open source [1.5] version became too slow in downloading .pom files. It worked well for years, but it starting taking more than 2 mins to download a .pom file of size 422 Bytes. But it used download few files faster. Manually we could have downloaded the same file either using browser, wget or curl in fractions of seconds. But artifact download through maven got affected badly.

We new the changes happened in our network. Our nexus host machine got moved to a restricted network, where access to internet was blocked. Finally we nailed down the issue is either maven/nexus is trying to contact external public repositories for updates for each download and slowing down things.


Issue resolution: Re-order group repositories.

By default with nexus installation, we had “Maven central” in top of public repositories.

Since Maven central is blocked in our network, nexus got slowed down.

Now I put our own hosted repositories on top of the “Ordered Group repositories”.

Now every search for artifact is getting matched in our local repository, instead of going external network and hence download got faster now.


Login to Nexus -> Public Repositories -> 

ssh command run from Jenkins shell will take “/” as home directory

Issue:

ssh command run from Jenkins shell will take “/” as home directory and hence it tries to look for all the files in that directory like “/.ssh/known_hosts” & /.ssh/id_rsa. It typically happens in Windows cygwin shell

Environment:

 Jenkins 

 Windows Slave

 Cygwin

 ssh

Error:    

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug2: no key of type 0 for host library-blr.ap.rsa.net

debug3: check_host_in_hostfile: filename /.ssh/known_hosts2

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts2

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug2: no key of type 2 for host library-blr.ap.rsa.net

Build was aborted

Fix:

1) Run Jenkins service as “Administrator”. By default it will be running  as "Local user"
              Start -> run -> services.msc -> Jenkins Slave -> Right click -> Properties -> Log on -> This account ->

           Give Administrator & password.

       

       Then in  jenkins -> manage nodes -> Select your machine -> Disconnect -> Then reconnect.

       Make sure the slave is launched as "Windows service"

        The above step should fix the issue.

     Note: Step 2 is optional. 

2) Run Cygwin-sshd as “Administrator.  By default it will be running  as “cyg_server”

Jenkins upgrade > 1.444
If you upgrade jenkins to latest versions, again ssh commands through windows will hang for the same reason mentioned above.
The fix is not to run Jenkins service as 'Local user'. Now you can configure it through jenkins node configuration page.
Login to jenkins -> Manage Jenkins -> Manage Nodes -> Click on your node -> configure -> Run service as -> select 'Log on using a different account' ->
For ex: 
User name:   .\Administrator  
pasword:  ********

Note:  .\  in user name is must , otherwise it fails with 
ERROR: Failed to create a service: Status Invalid Service Account

Thats it !!

Using BUILD_LOG_REGEX in jenkins email notification

Jenkins provide 'Email-ext' plugin, which allows to configure every aspect of email notifications.

One of my requirement is to send in mail the build package URL, which is already getting displayed in console output.

I found a token 

${BUILD_LOG_REGEX, regex, linesBefore, linesAfter, maxMatches, showTruncatedLines, substText, escapeHtml, matchedLineHtmlStyle} - Displays lines from the build log that match the regular expression. 

Here is example usage of it

${BUILD_LOG_REGEX, regex="^Build package URL:", showTruncatedLines=false}

This will extract the line starting with "Build package URL:" from the build log and send it through mail.

Using unix find Command to delete files older than a year

One of the boring task is to clean up data when ever disk usage exceeds the threshold limit. Among the many ways do it, here is the simple usage of Unix find command to delete files older than a year.

   find . -type f -mtime +365 -delete

Where -mtime +365 suggests to list files older than 365 days.
or
  find . -type f -atime +365 -delete

Where -atime stands for access time which is when the file was last read.


Some more examples

find . -mtime 0 # find files modified between now and 1 day ago 

# (i.e., within the past 24 hours)

find . -mtime -1 # find files modified less than 1 day ago 

# (i.e., within the past 24 hours, as before)

find . -mtime 1 # find files modified between 24 and 48 hours ago 

find . -mtime +1 # find files modified more than 48 hours ago

find . -mmin +5 -mmin -10 # find files modified between 

# 6 and 9 minutes ago

Usage of -exec option 
Here is the simple use case.
I want to find all the files with the name 'latest' in a directory and I want to run 'ls -l' command on the result, since I want to know whether 'latest' is a directory or soft link.
find . -maxdepth 2 -name latest -exec ls -l '{}' \;
Here the 
  -maxdepth option restrict the search to be conducted 2 levels of directory hierarchy.
The syntax '{}' \; is mandatory, which is tedious.

Unix find command to list files greater than 10MB
find . -size +10M -exec ls -lh '{}' \;

Unix find files older than 50 days and move them to another folder
find . -type f -mtime +50 -print| xargs -I {} mv {} /homes/siddesh/psbuild-portal-CGITemp-files

Using wget to download files recursively from https site

wget is a command used mostly in Linux to download files from web links (http, https or ftp). To download a file, a simple invocation of wget like
  wget http://mylibrary.com/build-kits/rel-2.7.1/prod1/prod1-2.7.1.10.tar
will download prod1-2.7.1.10.tar to the current directory.

To download set of files (ex .tar files) from a secured https site, we need to provide several options to it
  wget --no-check-certificate -r -l1 --no-parent -A.tar http://mylibrary.com/build-kits/rel-2.7.1/prod1/
where
  --no-check-certificate: don't validate the server's certificate
  -r or --recursive: specify recursive download
  -l1 or --level=NUMBER: maximum recursion depth (inf or 0 for infinite).
 --no-parent: don't ascend to the parent directory
 -A or --accept=LIST:  comma-separated list of accepted extensions.
 -nd or --no-directories: don't create directories.

Ant script to replace tokens (VERSION) in a file (build.properties)

Version of a product ever changes for main releases, SP releases, hotfix releases, etc. We can't hard-code version value in source files. One of the effective solution to tackle this is to use tokens in the place of version values and replace it before the start of build. The most common tool used to make string replacements are scripts written in perl/shell. Though scripts works well, but why to depend on scripts which are closely tied with Unix systems, to perform token replacement operations on platform independent Java build systems. 


 Java builds are usually managed through Ant/Maven. If perl/shell scripts are not stored along with Java build system, then if anyone checkouts only java code from source control, there build may fail due to improper version characters.

Here is an illustration of performing token replacement using Ant script.

Perforce sync is slow - Checklist for debugging

Is your Perforce performance is slow while doing sync or some other operations ?


1) Check you client spec’s view (if the repository is large) and if your client has a view of the entire repository, performance will be impacted (or you may have exceeded a maxRowCount error)
2) Modify your client spec (or create a new one) with a very narrow view (for testing, maybe down a single folder)
3) Do you have a firewall setup and preventing this kind of activity?
4) Can you try sync from another machine?
5)Try and play with "p4 ping" by progressively increasing the  message count, receive and transmit lengths (one at a time). If you hit the same wall repeatedly, then this could very well be indicative of some   throttling taking place somewhere
6) p4 -z tag info
7)  p4 -vN sync //depot/path/to/file      where N = 1,2,..9
8)  Network problems  http://kb.perforce.com/article/40/isolating-network-problems

 

Difference between centralized version control (CVCS) and distributed version control systems (DVCS)

1) Repository

    CVCS: Have a single server that contains all the versioned files and metadata 

    DVCS: Each users working copy of the codebase is a bona-fide repository.

2) Checkout

   CVCS: A number of clients check out files from central repository 

   DVCS: Clients don’t just check out the latest snapshot of the files: they fully mirror the repository

3) Tracking latest revision

   CVCS: Everyone knows to a certain degree what everyone else on the project is doing. The trunk usually has the latest revision.

   DVCS: It is not plainly obvious.

Comparison of several SCM tools

Here is a link which compares many SCM tools inlcuding Distributed and Central version control tools.
It compares for few use cases, anyways take a look at http://better-scm.shlomifish.org/comparison/comparison.html

Jenkins - FATAL: Unable to produce a script file

Immediately after syncing code from perforce, a job in jenkins fails with the error message:


09:16:33  Sync complete, took 8001 ms
09:16:33  No emails were triggered.
09:16:33  FATAL: Unable to produce a script file
09:16:33  hudson.util.IOException2: Failed to create a temp file on /export/jenkins/jobs/jenkins-blr-backup/workspace

Fix: Restarted the jenkins fixed the issue
Root cause:  No idea !!!!

A build engineer's use-case of java decompiler

This is one scenario, which demonstrate the usage of java decompiler to a build engineer.

One morning, a developer walked to my cube and complained
    Developer: My changes are not picked in latest build
As a build engineer, it's a familiar complaint which I usually receive.
    Me: Cool !! provide me your change location in Perforce SCM tool and some time to debug.
That's the end of my short conversation. Now I started a massive search operation to find out, why the pretty girls changes are not picked :)

It's a Java based build system using Maven 2 and Jenkins.

Step 1: I made sure the code changes are picked up in last build by verifying the Change-list number available in perforce and the one Jenkins has picked during the build. Even I verified in the build machine, by checking the files it has synced. Everything is perfect, then I started wondering what's going wrong.

Step 2:  Since it is a maven build, I thought to check the content of jar package, our build has uploaded to nexus repository. I downloaded and extracted it and the got the mystery .class file. Now how can I prove that her changes are picked by the build using this .class file?

Step 3: I felt the need of a java decompiler. I downloaded "cavaj" java decompiler (it's free) and installed it. Opened the mystery .class file with java decompiler and I'm glad to find her changes.

Java decompiler is used to convert Java class files into Java source code. In fact, there is an arms race brewing between decompilers and so-called obfuscators, which profess to provide Java code some measure of protection from decompilers. In essence, obfuscators remove all non-essential symbolic information from your class files and, optionally, replace it with fake symbolic information designed to confuse the decompiler


Step 4: Now for curiosity I extended further to find out why the developer complaining it is not picked. The root cause of the issue is that,  developer was validating a jar file, which consumes her changes from another jar file. It's a maven dependency issue. Some how, the final jar file was not picking the right version of the dependent jar file, which is fixed later.

Jira Administration quick reference

1) Configuring the Layout and Design
Customising the Look and Feel
    Administration -> Global Settings -> Look and Feel -> To edit the logo, colours or time format, click the 'Edit Configuration' link at the bottom of the page
Changing the default language
  Administration -> General Configuration -> Edit Configuration > select the appropriate language in the drop-down box next to 'Default language'.
Configuring default issue navigator
  The Issue Navigator is used within JIRA to find and filter issues, and to display the search results in various formats ('views'). It is possible to

select which issue fields will be displayed as columns in the Issue Navigator.

   Administration -> Issue Fields -> Navigator Columns -> Add new column
Configuring the Default Dashboard
  Administration -> Global Settings -> System Dashboard -> Add Gadget
Configuring an Announcement Banner
  Administration -> Options & Settings -> Announcement Banner
Enabling Logout Confirmation
  Global Settings -> General Configuration -> Edit Configuration -> Logout confirmation

Productivity tools linkage

Reference: From slide prepared by Carl Sjoquist. 

I liked this image which depicts how various productivity tools in an organization collaborate. 

Generating ccs (Conary Change Set) file using rmake build Job-ID

To generate CCS (Conary Change Set) files, for the corresponding packages available on rBuilder server, follow these steps

• rmake build <package name>.recipe

             This command will build the specified recipe file

             Ex: rmake build tomcat.recipe

          Note down the job number 

• rmake changeset <jobid>  <package name>.ccs

          This command creates a changeset for the given <jobId>. The created changeset contains all the binaries built for this job.

          Ex: rmake changeset 573 tomcat.ccs 

• conary showcs --lsl <package name>.ccs

           Use conary showcs to examine changeset file contents 

            Ex: conary showcs --lsl tomcat.ccs 

Cygwin error: Couldn't create pipe tracker for pid

Our builds started failing to connect to a Windows machine using sshd provided by cygwin.
Then when I tried to open cygwin shell, it started throwing the error

pinfo::wait: Couldn't create pipe tracker for pid 3534

Win32 error 231
bash: fork: Resource temporarily unavailable

After googling, the reason for this error, which I came to know is
It's a limitation in the number of running processes under Windows.  When cygwin cannot create a new process during a  fork(), you will get this behavior. 
Some people have reported success turning off (or swapping) antivirus programs, or other tricks to reduce the number of running processes on their system.
 
Well, what that message means is that for some reason, the child process
failed to notify the parent of its existence within a reasonable amount of
time after it was spawned.  That could mean that it died for some reason, or
it could mean that it was alive, but failed to notify the parent process.


Then I felt, it's good to reboot my Windows machine once and it has solved this problem for the time being. I will keep updated if it pop up again.

Our cygwin version is
$ cygcheck -V
cygcheck version 1.90.4.1
System Checker for Cygwin
Copyright 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Red Hat, Inc.
Compiled on Jun 12 2008

and Windows version is "Windows XP professional 2002 SP2" 

I guess this issue doesn't occur with latest cygwin/Windows versions.

nexus "Item not found on path" error

We are using nexus 1.5.0 repository to host maven packages in our organization.
Recently I started encountering the error like
   Item not found on path "/com/rsa/appliance/appliance-distribution/3.1.3-SNAPSHOT/"!
when I click to download this package from nexus GUI, even though a maven build has successfully uploaded this package to the server, which I verified by logging into the machine which is hosting nexus (a RHEL m/c)

Issue:

The Not Found Cache is getting populated with these artifacts. Need to clear the cache for the repo and try the request again. 


Solution:
 Login to Nexus as Admin and browse -> Repositories -> Right click on repository which is giving the issue (in our case it's snapshots) -> Expire Cache

This has solved our issue.

Encrypting using GPG - an example

GPG stands for Gnu Privacy Guard, is a tool used to sign and encrypt data. Here I'm giving a simple usage of this tool.

Usecase: We have a file on linux server, which needs to be encrypted and delivered to another machine.

Solution:

• By default every linux server comes with "gpg" tool. You can verify by running the command "which gpg"
• Use gpg command to encrypt your file

           cd TO_YOUR_FILE_LOCATION   (I need to encrypt a file named CONARY)

           

           gpg -c CONARY           (It will ask for the passphrase/password, give a strong one and remember it)

           It creates a file named CONARY.gpg, which is encrypted. You can take a look at the content of this file

• Using gpg command for decryption

          Now I have transferred this encrypted file (CONARY.gpg) to another server where gpg utility is available and run below command to decrypt it

         gpg -o CONARY -d CONARY.gpg

      

          Where -o option tells to create a file CONARY, instead of printing output on console


Strengthening encryption using AES256 cipher algo


Command to Encrypt
gpg --symmetric --cipher-algo aes256 -o dpm-3.2-src.tar.gz.gpg dpm-3.2-src.tar.gz
Enter passphrase: *******

Repeat passphrase: *******

Command to decrypt
gpg -d -o dpm-3.2-src.tar.gz  dpm-3.2-src.tar.gz.gpg


gpg: AES256 encrypted data
Enter passphrase: *******

         

                 

         

           

how to check open ports on linux machine?

Use the commands
1) nmap -sS -O 127.0.0.1 or
2) netstat -nap   
   netstat --listen
 To display open ports and established TCP connections, enter:
   netstat -vatn
 To display only open UDP ports try the following command:
  netstat -vaun\
  if you want to see FQDN (full dns hostname), try removing the -n flag:
  netstat -vat

Jenkins most useful features

Jenkins is a CI (Continuous Integration) tool which is used to widely to manage builds and tests. In this blog, I'm listing some of the add on's which makes Jenkins richer and support it's use to manage builds using Jenkins instead of homegrown scripts. Most of these additional functionality comes through Jenkins plugin's.

1) Support to GIT source code management
    Plugin: Jenkins GIT plugin
    It checkout code from GIT repository (using http/ssh/local/git protocol), poll for changes and trigger build

2) Archive the build artifacts (Post build section)
    Provides link for download to artifacts which are built. It acts a one stop dashboard where you build and distribute artifacts to consumers. These artifacts are preserved as long as logs are preserved.
   Plugin: Comes with default Jenkins download

3) Publishing Junit test result on build dashboard (Post build section)
    If your project running junit tests, provide link to xml reports. Jenkins publish the test results , compose historical trends, web UI for viewing/tracking test results.
   Plugin: Comes with default Jenkins download

4) Publishing Javadocs on build dashboard (Post build section)
   If your project generating javadocs (for ex: mvn javadoc:javadoc), provide link to xml reports. Jenkins will publish the latest Javadoc and it can also retain older javadocs if needed.
   Plugin: Comes with default Jenkins download

5) Publishing code coverage metrics ( ex: cobertura )
   If your project calculating code coverage metrics (for ex: mvn cobertura:cobertura), provide link to xml reports generated by your coverage tools like cobertura/clover/Emma. Jenkins will publish the coverage metrics on job dashboard.
     Plugin: Jenkins cobertura plugin
  Cobertura plugin lets you to fine-tune minimum acceptable levels of code coverage.

6) Build trends
   Clicking "trend" for a project displays build time trend, timeline graph (scroll your mouse on graph to see the historical build timings).