The presentation secrets of steve jobs - Carmine Gallo

The Presentation Secrets of Steve Jobs - Carmine Gallo from Carmine Gallo

How to customize MANIFEST files in WAR using Maven?

A war file contains MANIFEST files which is created by Maven. This post explains how to customize a manifest files to add our own values, which may get reflected while deploying on application servers like websphere.

The post How to create java war (web archive) file using Maven?  explains in detail about building war file for sample application using Maven. Refer it for creating your own simple webapp. This post is the continuation to it.

When you explode the war file created for the sample application code described in my previous post, it contains the below directory structure.

   simple-1.1-SNAPSHOT

       images/springsource.png

       jsp/hello.jsp

       META-INF

           maven/com.rsa.siddesh.simple/simple

              pom.properties

              pom.xml

           MANIFEST.MF

     WEB-INF

          web.xml

          classes

            examples/Hello.class

            images/springsource.png

          lib/servlet-api-2.5.jar

       index.html

The default MANIFEST.MF created by Maven looks like this

MANIFEST.MF

   Manifest-Version: 1.0

   Archiver-Version: Plexus Archiver

   Created-By: Apache Maven

   Built-By: guruss1

   Build-Jdk: 1.6.0_16

We can add many values to it through Maven.

1) 

How to create java war (web archive) file using Maven?

Maven by default creates a JAR package. But we can build package in any other format easily through Maven. This blog explains how to generate a WAR package, customizing the MANIFEST and web.xml files within it, using simple HelloWorld example.

Project structure
<proj-home>
        pom.xml
        src
          main
            java
               App.java
        target
           simple-1.1-SNAPSHOT.jar


First we create a Java file which prints Hello World.
  Maven projects expects Java source files under  src/main/java directory. Hence lets create our Hellow World program App.java under it.

The content of App.java is

Upgrading Sonatype nexus from 1.5 to 1.6

Here are the steps followed while upgrading Sonatype nexus server from 1.5 to 1.6 version in Linux server (RHEL)

Jira workflows - Dealing with validator

Recently I got a request to make certain fields mandatory while marking a issue as 'Fix' (A step in our workflow'). Initially I made the requested custom fields as 'Required' in Field configuration, but it screwed !!!. After this change it started asking to provide values for fields which are not visible in 'create' page.
Finally I learnt it needs to be configured in Workflows validator.
I'm sharing my learning's here.

Introduction
Workflow is the place where we define business process. A JIRA workflow is the set of steps (or statuses) and transitions that an issue goes through during its lifecycle.
The below picture represents the default Jira workflow.

JIRA workflows consist of steps and transitions:
  A step represents a workflow's current status for an issue.An issue can exist in one step only at any point in time. In the diagram above the rectangular boxes represent steps/statuses.
 A transition is a link between two steps. A transition allows an issue to move from one step to another step. A transition is a one-way link, so if an issue needs to move back and forth between two steps, two transitions need to be created. In the diagram above the arrows represent transitions.

jenkins nested views not displaying with read permissions after upgrade

Today I upgraded Jenkins from 1.444 version to 1.470 and to my surprise it was not showing any nested views for anonymous users. It was showing only the default 'All' view. But if we login to jenkins, then it shows all nested views.

When I googled I found this relevant link https://issues.jenkins-ci.org/browse/JENKINS-13429

As suggested I had already had Jenkins version > 1.459, Nested View Plugin 1.8. But I didn't had Role-based Authorization Strategy. Even I added this plugin to my jenkins.

Then I realized that this issue can fixed without adding Role based Authorization Strategy.

We are already using 'Project-based Matrix Authorization Strategy' and migrating to Role based authorization will be some additional tasks, which I didn't wanted to do.

Anyways the solution is enable 'Anonymous' account 'Read' 'View' in 'Project-based Matrix Authorization Strategy'

Here is the detailed way of doing it

Login to Jenkins -> Manage Jenkins -> Configure System -> Access Control -> Authorization -> Project-based Matrix Authorization Strategy

For Anonymous user -> View section -> Click 'Read'

How to run dos commands from cygwin? File name too long - fix

Typically we write shell scripts and run it under cygwin shell. The command used in scripts like rm, mv are obtained from cygwin installation. But recently we encountered a tedious issue with cygwin shell.
The 'rm' command run from cygwin fails with message "File name too long" for some of the files and it's causing build failure.

After a bit of research, I found there is a limit for file name length in cygwin (I guess 256 characters). Upgrading the cygwin may fix the issue, but I didn't want to do that.

Fix: I found that the native dos 'rmdir' command can delete these lengthy file names without any issue and hence I decided to call native dos commands from Shell scripts.

Here is a way to call native dos commands from cygwin shell
  cmd /C rmdir /S /Q <directory>


here
   cmd   -  Starts a new instance of the Windows command interpreter
       /C   - Carries out the command specified by string and then terminates

  rmdir - dos command to delete directory
     /S - recursive delete of files & sub-directories
     /Q  - Quiet


Just add this command to shell script. It may need some if condition to run this only on Windows platform.

rsync - Syncing soft-link's as soft-link

We were running rsync between 2 repositories (created on Linux OS) from many days. But we missed to handle soft-links properly. The option 'l' provided by rsync will update the soft-links in the destination server accordingly. If this option is not provided, in the destination, the directory pointed by source will be created with the soft-link name and it will be never updated at all !


Here is a usage of rsync
/usr/bin/rsync -avuzl --stats /export/kits/dpm/builds/dev/rkm root@us-repo.org:/export/kits/dpm/builds/dev


Where the options
 -a, --archive               archive mode;
 -v, --verbose               increase verbosity
 -u, --update                skip files that are newer on the receiver
 -z, --compress              compress file data during the transfer
 -l, --links                 copy symlinks as symlinks

Deploying maven build artifacts to Nexus repository

To deploy the artifacts generated by your maven build to nexus repository, you need configure it in 2 files
1) Parent pom.xml
2) settings.xml

and also to prevent publishing password in settings.xml file, you need to configure password less ssh connection between your build machine and nexus repository


Pom.xml changes
Add distributionManagement section to your parent pom.xml and provide ID, name, url for various delivery types like snapshots, releases, site, etc.
Here is an example from our project

Jira Administration - Custom Fields & Screens

Jira has a complex administration concepts. Performing Jira admin tasks without understanding the complete flow leads to mistakes. In this post, I'm explaining how the Custom Fields are created and how to link it to project screens.

Creating custom fields
Let's assume you get a request to create a custom field "Severity". Check whether this custom field is already existing by

• Login to Jira -> Administration -> Issue Fields -> Custom Fields

If it is not available, create it straight away

• Login to Jira -> Administration -> Issue Fields -> Custom Fields -> Add custom Field -> Choose the field type -> Provide field name, search template, etc , applicable issue types, Applicable context.

   Custom Field Context

        The custom field which we created can be associated to all issues by selecting "Global context" or to specific projects. Depending on the usability of the custom field make it either global or project specific.

If the custom field is already exist, then try to reuse it.

Screens

Screens group multiple issue fields. Using Screens, you can control which fields are displayed. You can also split fields on a Screen into multiple tabs.

Issue: Nexus is too slow in downloading pom files

Recently our nexus open source [1.5] version became too slow in downloading .pom files. It worked well for years, but it starting taking more than 2 mins to download a .pom file of size 422 Bytes. But it used download few files faster. Manually we could have downloaded the same file either using browser, wget or curl in fractions of seconds. But artifact download through maven got affected badly.

We new the changes happened in our network. Our nexus host machine got moved to a restricted network, where access to internet was blocked. Finally we nailed down the issue is either maven/nexus is trying to contact external public repositories for updates for each download and slowing down things.


Issue resolution: Re-order group repositories.

By default with nexus installation, we had “Maven central” in top of public repositories.

Since Maven central is blocked in our network, nexus got slowed down.

Now I put our own hosted repositories on top of the “Ordered Group repositories”.

Now every search for artifact is getting matched in our local repository, instead of going external network and hence download got faster now.


Login to Nexus -> Public Repositories -> 

ssh command run from Jenkins shell will take “/” as home directory

Issue:

ssh command run from Jenkins shell will take “/” as home directory and hence it tries to look for all the files in that directory like “/.ssh/known_hosts” & /.ssh/id_rsa. It typically happens in Windows cygwin shell

Environment:

 Jenkins 

 Windows Slave

 Cygwin

 ssh

Error:    

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug2: no key of type 0 for host library-blr.ap.rsa.net

debug3: check_host_in_hostfile: filename /.ssh/known_hosts2

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts2

debug3: check_host_in_hostfile: filename /.ssh/known_hosts

debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts

debug2: no key of type 2 for host library-blr.ap.rsa.net

Build was aborted

Fix:

1) Run Jenkins service as “Administrator”. By default it will be running  as "Local user"
              Start -> run -> services.msc -> Jenkins Slave -> Right click -> Properties -> Log on -> This account ->

           Give Administrator & password.

       

       Then in  jenkins -> manage nodes -> Select your machine -> Disconnect -> Then reconnect.

       Make sure the slave is launched as "Windows service"

        The above step should fix the issue.

     Note: Step 2 is optional. 

2) Run Cygwin-sshd as “Administrator.  By default it will be running  as “cyg_server”

Jenkins upgrade > 1.444
If you upgrade jenkins to latest versions, again ssh commands through windows will hang for the same reason mentioned above.
The fix is not to run Jenkins service as 'Local user'. Now you can configure it through jenkins node configuration page.
Login to jenkins -> Manage Jenkins -> Manage Nodes -> Click on your node -> configure -> Run service as -> select 'Log on using a different account' ->
For ex: 
User name:   .\Administrator  
pasword:  ********

Note:  .\  in user name is must , otherwise it fails with 
ERROR: Failed to create a service: Status Invalid Service Account

Thats it !!

Using BUILD_LOG_REGEX in jenkins email notification

Jenkins provide 'Email-ext' plugin, which allows to configure every aspect of email notifications.

One of my requirement is to send in mail the build package URL, which is already getting displayed in console output.

I found a token 

${BUILD_LOG_REGEX, regex, linesBefore, linesAfter, maxMatches, showTruncatedLines, substText, escapeHtml, matchedLineHtmlStyle} - Displays lines from the build log that match the regular expression. 

Here is example usage of it

${BUILD_LOG_REGEX, regex="^Build package URL:", showTruncatedLines=false}

This will extract the line starting with "Build package URL:" from the build log and send it through mail.

Using unix find Command to delete files older than a year

One of the boring task is to clean up data when ever disk usage exceeds the threshold limit. Among the many ways do it, here is the simple usage of Unix find command to delete files older than a year.

   find . -type f -mtime +365 -delete

Where -mtime +365 suggests to list files older than 365 days.
or
  find . -type f -atime +365 -delete

Where -atime stands for access time which is when the file was last read.


Some more examples

find . -mtime 0 # find files modified between now and 1 day ago 

# (i.e., within the past 24 hours)

find . -mtime -1 # find files modified less than 1 day ago 

# (i.e., within the past 24 hours, as before)

find . -mtime 1 # find files modified between 24 and 48 hours ago 

find . -mtime +1 # find files modified more than 48 hours ago

find . -mmin +5 -mmin -10 # find files modified between 

# 6 and 9 minutes ago

Usage of -exec option 
Here is the simple use case.
I want to find all the files with the name 'latest' in a directory and I want to run 'ls -l' command on the result, since I want to know whether 'latest' is a directory or soft link.
find . -maxdepth 2 -name latest -exec ls -l '{}' \;
Here the 
  -maxdepth option restrict the search to be conducted 2 levels of directory hierarchy.
The syntax '{}' \; is mandatory, which is tedious.

Unix find command to list files greater than 10MB
find . -size +10M -exec ls -lh '{}' \;

Unix find files older than 50 days and move them to another folder
find . -type f -mtime +50 -print| xargs -I {} mv {} /homes/siddesh/psbuild-portal-CGITemp-files

Using wget to download files recursively from https site

wget is a command used mostly in Linux to download files from web links (http, https or ftp). To download a file, a simple invocation of wget like
  wget http://mylibrary.com/build-kits/rel-2.7.1/prod1/prod1-2.7.1.10.tar
will download prod1-2.7.1.10.tar to the current directory.

To download set of files (ex .tar files) from a secured https site, we need to provide several options to it
  wget --no-check-certificate -r -l1 --no-parent -A.tar http://mylibrary.com/build-kits/rel-2.7.1/prod1/
where
  --no-check-certificate: don't validate the server's certificate
  -r or --recursive: specify recursive download
  -l1 or --level=NUMBER: maximum recursion depth (inf or 0 for infinite).
 --no-parent: don't ascend to the parent directory
 -A or --accept=LIST:  comma-separated list of accepted extensions.
 -nd or --no-directories: don't create directories.

Ant script to replace tokens (VERSION) in a file (build.properties)

Version of a product ever changes for main releases, SP releases, hotfix releases, etc. We can't hard-code version value in source files. One of the effective solution to tackle this is to use tokens in the place of version values and replace it before the start of build. The most common tool used to make string replacements are scripts written in perl/shell. Though scripts works well, but why to depend on scripts which are closely tied with Unix systems, to perform token replacement operations on platform independent Java build systems. 


 Java builds are usually managed through Ant/Maven. If perl/shell scripts are not stored along with Java build system, then if anyone checkouts only java code from source control, there build may fail due to improper version characters.

Here is an illustration of performing token replacement using Ant script.

Perforce sync is slow - Checklist for debugging

Is your Perforce performance is slow while doing sync or some other operations ?


1) Check you client spec’s view (if the repository is large) and if your client has a view of the entire repository, performance will be impacted (or you may have exceeded a maxRowCount error)
2) Modify your client spec (or create a new one) with a very narrow view (for testing, maybe down a single folder)
3) Do you have a firewall setup and preventing this kind of activity?
4) Can you try sync from another machine?
5)Try and play with "p4 ping" by progressively increasing the  message count, receive and transmit lengths (one at a time). If you hit the same wall repeatedly, then this could very well be indicative of some   throttling taking place somewhere
6) p4 -z tag info
7)  p4 -vN sync //depot/path/to/file      where N = 1,2,..9
8)  Network problems  http://kb.perforce.com/article/40/isolating-network-problems

 

Difference between centralized version control (CVCS) and distributed version control systems (DVCS)

1) Repository

    CVCS: Have a single server that contains all the versioned files and metadata 

    DVCS: Each users working copy of the codebase is a bona-fide repository.

2) Checkout

   CVCS: A number of clients check out files from central repository 

   DVCS: Clients don’t just check out the latest snapshot of the files: they fully mirror the repository

3) Tracking latest revision

   CVCS: Everyone knows to a certain degree what everyone else on the project is doing. The trunk usually has the latest revision.

   DVCS: It is not plainly obvious.

Comparison of several SCM tools

Here is a link which compares many SCM tools inlcuding Distributed and Central version control tools.
It compares for few use cases, anyways take a look at http://better-scm.shlomifish.org/comparison/comparison.html

Jenkins - FATAL: Unable to produce a script file

Immediately after syncing code from perforce, a job in jenkins fails with the error message:


09:16:33  Sync complete, took 8001 ms
09:16:33  No emails were triggered.
09:16:33  FATAL: Unable to produce a script file
09:16:33  hudson.util.IOException2: Failed to create a temp file on /export/jenkins/jobs/jenkins-blr-backup/workspace

Fix: Restarted the jenkins fixed the issue
Root cause:  No idea !!!!