Showing posts with label jenkins-hudson. Show all posts
Showing posts with label jenkins-hudson. Show all posts

Wednesday, May 17, 2017

Jenkins groovy script to find the node on which the last build built for a job

The below code is intended to be run in Jenkins script console

Problem

I want to find the jenkins node on which the last build of the job "my-job-name" built. This script determined that, it ran on centos5-x64-01

Code

def getJobs() {
 def hi = hudson.model.Hudson.instance
 return hi.getItems(hudson.model.Job)
}

def getBuildJob(String jobNam) {
 def buildJob = null
 def jobs = getJobs()
 (jobs).each { job ->
 if (job.displayName == jobNam) {
 println("Found")
 println("Exiting job search")
 buildJob = job
 return buildJob
 }
 }
 return buildJob
}

job_name = "my-job-name"

job_id = getBuildJob(job_name)

last_build = job_id.getBuilds()[0]

println("Recent Builds of the job " + job_name + " " + job_id.getBuilds())
println("last_build="+last_build)

println("The last_build of Job " + job_name + " builton the node " +last_build.getBuiltOn())


Output

Found
Exiting job search
Recent Builds of the job my-job-name [my-job-name #1474, my-job-name #1473, my-job-name #1472, my-job-name #1466, my-job-name #1421]
last_build=my-job-name #1474
The last_build of Job my-job-name builton the node hudson.slaves.DumbSlave[centos5-x64-01]

Reference

Jenkins groovy script to list busy nodes

The below code is intended to be run in Jenkins script console

def busy = [];
busy = hudson.model.Hudson.instance.slaves.findAll { it.getComputer().countBusy() > 0 }
out.println("Busy nodes: "+busy.collect { it.nodeName }.join(", "))

Sunday, May 14, 2017

Jenkins Elastic Build slaves with Apache Mesos

In this blog, I'll share my experience of setting up Jenkins server to obtain slaves dynamically from Mesos. You can get the broader picture of this, from the reference links provided in the end.
This is a very simple test setup. Basically install Jenkins, Mesos on my Mac laptop and run a simple hello-world jenkins job, which will get executed on a mesos cloud.

My Environment

  • Mesos - 1.20
  • Jenkins - 2.46.2
  • Mac OS - Sierra (10.12.3)

Installation

  • Mesos - There is no pre-built installer. We need to get the code and build it. Refer http://mesos.apache.org/documentation/latest/getting-started/ for complete details. Below are the commands I ran on my MacBook to install mesos
    • Download mesos code
    • Extract
      • tar -xvfz mesos-1.2.0.tar.gz
    • Install mesos build pre-requisites
      • brew install Caskroom/cask/java (I already had java and hence didn't run this)
      • brew install wget git autoconf automake libtool subversion maven
        • Got errors with git, subversion, maven, since I already had old version. It asked me to upgrade. Upgraded them with below command
        • brew upgrade git
        • brew upgrade maven
        • As per mesos doc, chose to uninstall existing subversion and re-install
          • brew unlink subversion
          • brew install subversion
      • I had python pip & virtualenv and hence I skipped installing them
    • Build the mesos code
      • cd mesos-1.2.0
      • ./bootstrap
      • ./configure CXXFLAGS=-Wno-deprecated-declarations
      • make
      • make check
    • Start Mesos master
      • sudo su -
      • mkdir /var/lib/mesos
      • ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos
    • Start mesos agent
      • Open new terminal
      • cd mesos-1.2.0
      • ./bin/mesos-agent.sh --master=127.0.0.1:5050 --work_dir=/var/lib/mesos
    • Test/Verify your mesos
      • Browse http://127.0.0.1:5050
      • Run test frameworks
        • C++:   ./src/test-framework --master=127.0.0.1:5050
        • Java: ./src/examples/java/test-framework 127.0.0.1:5050
        • Python: ./src/examples/python/test-framework 127.0.0.1:5050 . It gave me error and I didn't bother to troubleshoot it
  • Installing Jenkins
    • Download jenkins-*.pkg file and install it on your laptop
    • Refer https://jenkins.io/download/
  • Install Jenkins mesos plugin
    • Browse your local jenkins : http://localhost:8080 -> Manage Jenkins -> Manage plugins -> Available -> Search Mesos -> Install and reboot jenkins (if required)
  • Configure mesos details in your Jenkins
    • Browse to http://localhost:8080/configure -> Cloud -> Add a new cloud
    • Give the mesos client binary path under "Mesos native library path". Since I have built mesos in Mac, the shared library will be "libmesos.dylib". For linux it will be libmesos.so
    • Provide the "Mesos Master [hostname:port]", Description and retain the rest of defaults. By default the mesos host label will be "mesos". You need to use this label while configuring job
    • Test the mesos connection by clicking "Test Connection"

Configure Jenkins job to use mesos

  • Create a simple HelloWorld Freestyle project
  • Importantly, you need to specify the label "mesos" under "Restrict where this project can be run"

Test

  • Run the HelloWorld project
  • As you can observe, this job got executed on a mesos host mesos-jenkins-ddd4afdc45d7490c80a9706889044586-mesos

Reference

Tuesday, December 13, 2016

Programming Jenkins jobs with Job DSL

The Jenkins job configuration can be programmed using Job DSL, instead of manually configuring it for the following advantages

  • Job configuration can be version controlled under tools like GIT.
  • Easily apply modifications to numerous existing jobs, by changing code and re-running DSL
  • It provides the centralised location to view and analyse all Jenkins job config information
What you need to start with Job DSL?
  • Job DSL Plugin
  • Little bit of Groovy language knowledge

Job DSL Plugin

Install Job DSL plugin and then you'll get a new build step, 'Process Job DSLs' under 'Freestyle project'

Usage

You can run DSL code either

1) By providing DSL groovy code directly at 'DSL Script' section. This option is useful while testing your DSL code. In the below screen shot contains DSL instruction for the job named 'build-hyperloop'

 

2) or Store your DSL code in a groovy file (for ex. myjob.groovy) and ensure it is available in the job workspace. Select 'Look on Filesystem' option and provide the location of this groovy code at 'DSL Scripts'

 

Now save and 'Build' this job and it will create a new Jenkins job by name 'build-hyperloop' if not exist or if this job already exists, then it updates the job with this configuration

End to End flow

Monday, October 29, 2012

Jenkins Failed to locate Cygwin installation. Is Cygwin installed?


You configured a new Windows  (64 bit) node in Jenkins  (either run as Windows servvice or JNLP or ssh) and when you try to execute a shell command from it, you may get the below mentioned error.

FATAL: command execution failedhudson.util.IOException2: Failed to locate Cygwin installation. Is Cygwin installed? at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.getCygwinRoot(CygpathLauncherDecorator.java:122) at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.call(CygpathLauncherDecorator.java:127) at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.call(CygpathLauncherDecorator.java:112) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:287) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)Caused by: hudson.util.jna.JnaException: Win32 error: 2 - null at hudson.util.jna.RegistryKey.check(RegistryKey.java:124) at hudson.util.jna.RegistryKey.open(RegistryKey.java:223) at hudson.util.jna.RegistryKey.openReadonly(RegistryKey.java:218) at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.getCygwinRoot(CygpathLauncherDecorator.java:115) ... 11 more

You can try the fix provided in the link https://issues.jenkins-ci.org/browse/JENKINS-6992

If you are still getting the same issue, then check in your Jenkins shell, whether you have
#!/bin/bash
in the first line.
If you have this, remove it. Your shell script will work fine.

Friday, July 6, 2012

jenkins nested views not displaying with read permissions after upgrade

Today I upgraded Jenkins from 1.444 version to 1.470 and to my surprise it was not showing any nested views for anonymous users. It was showing only the default 'All' view. But if we login to jenkins, then it shows all nested views.
When I googled I found this relevant link https://issues.jenkins-ci.org/browse/JENKINS-13429
As suggested I had already had Jenkins version > 1.459, Nested View Plugin 1.8. But I didn't had Role-based Authorization Strategy. Even I added this plugin to my jenkins.

Then I realized that this issue can fixed without adding Role based Authorization Strategy.
We are already using 'Project-based Matrix Authorization Strategy' and migrating to Role based authorization will be some additional tasks, which I didn't wanted to do.

Anyways the solution is enable 'Anonymous' account 'Read' 'View' in 'Project-based Matrix Authorization Strategy'

Here is the detailed way of doing it

Login to Jenkins -> Manage Jenkins -> Configure System -> Access Control -> Authorization -> Project-based Matrix Authorization Strategy
For Anonymous user -> View section -> Click 'Read'





Tuesday, May 15, 2012

ssh command run from Jenkins shell will take “/” as home directory

Issue:
ssh command run from Jenkins shell will take “/” as home directory and hence it tries to look for all the files in that directory like “/.ssh/known_hosts” & /.ssh/id_rsa. It typically happens in Windows cygwin shell

Environment:
 Jenkins 
 Windows Slave
 Cygwin
 ssh


Error:    
debug3: check_host_in_hostfile: filename /.ssh/known_hosts
debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts
debug3: check_host_in_hostfile: filename /.ssh/known_hosts
debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts
debug3: check_host_in_hostfile: filename /.ssh/known_hosts
debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts
debug2: no key of type 0 for host library-blr.ap.rsa.net
debug3: check_host_in_hostfile: filename /.ssh/known_hosts2
debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts2
debug3: check_host_in_hostfile: filename /.ssh/known_hosts
debug3: check_host_in_hostfile: filename /etc/ssh_known_hosts
debug2: no key of type 2 for host library-blr.ap.rsa.net
Build was aborted

Fix:

1) Run Jenkins service as “Administrator”. By default it will be running  as "Local user"
              Start -> run -> services.msc -> Jenkins Slave -> Right click -> Properties -> Log on -> This account ->
           Give Administrator & password.
       
       Then in  jenkins -> manage nodes -> Select your machine -> Disconnect -> Then reconnect.
       Make sure the slave is launched as "Windows service"

        The above step should fix the issue.

     Note: Step 2 is optional. 

2) Run Cygwin-sshd as “Administrator.  By default it will be running  as “cyg_server”

Jenkins upgrade > 1.444
If you upgrade jenkins to latest versions, again ssh commands through windows will hang for the same reason mentioned above.
The fix is not to run Jenkins service as 'Local user'. Now you can configure it through jenkins node configuration page.
Login to jenkins -> Manage Jenkins -> Manage Nodes -> Click on your node -> configure -> Run service as -> select 'Log on using a different account' ->
For ex: 
User name:   .\Administrator  
pasword:  ********

Note:  .\  in user name is must , otherwise it fails with 
ERROR: Failed to create a service: Status Invalid Service Account




Thats it !!



Monday, February 13, 2012

Jenkins - FATAL: Unable to produce a script file

Immediately after syncing code from perforce, a job in jenkins fails with the error message:


09:16:33  Sync complete, took 8001 ms
09:16:33  No emails were triggered.
09:16:33  FATAL: Unable to produce a script file
09:16:33  hudson.util.IOException2: Failed to create a temp file on /export/jenkins/jobs/jenkins-blr-backup/workspace

Fix: Restarted the jenkins fixed the issue
Root cause:  No idea !!!!

Friday, December 16, 2011

Jenkins most useful features

Jenkins is a CI (Continuous Integration) tool which is used to widely to manage builds and tests. In this blog, I'm listing some of the add on's which makes Jenkins richer and support it's use to manage builds using Jenkins instead of homegrown scripts. Most of these additional functionality comes through Jenkins plugin's.

1) Support to GIT source code management
    Plugin: Jenkins GIT plugin
    It checkout code from GIT repository (using http/ssh/local/git protocol), poll for changes and trigger build

2) Archive the build artifacts (Post build section)
    Provides link for download to artifacts which are built. It acts a one stop dashboard where you build and distribute artifacts to consumers. These artifacts are preserved as long as logs are preserved.
   Plugin: Comes with default Jenkins download

3) Publishing Junit test result on build dashboard (Post build section)
    If your project running junit tests, provide link to xml reports. Jenkins publish the test results , compose historical trends, web UI for viewing/tracking test results.
   Plugin: Comes with default Jenkins download

4) Publishing Javadocs on build dashboard (Post build section)
   If your project generating javadocs (for ex: mvn javadoc:javadoc), provide link to xml reports. Jenkins will publish the latest Javadoc and it can also retain older javadocs if needed.
   Plugin: Comes with default Jenkins download

5) Publishing code coverage metrics ( ex: cobertura )
   If your project calculating code coverage metrics (for ex: mvn cobertura:cobertura), provide link to xml reports generated by your coverage tools like cobertura/clover/Emma. Jenkins will publish the coverage metrics on job dashboard.
     Plugin: Jenkins cobertura plugin
  Cobertura plugin lets you to fine-tune minimum acceptable levels of code coverage.

6) Build trends
   Clicking "trend" for a project displays build time trend, timeline graph (scroll your mouse on graph to see the historical build timings).

Thursday, December 15, 2011

jenkins error: java.io.IOException: Authentication method password not supported by the server at this stage

When I tried to add a node to jenkins/hudson using ssh as launch method, the authentication keeps on failing with the below error.


[12/15/11 13:23:12] [SSH] Opening SSH connection to **.**.***.**:22.
[12/15/11 13:23:12] [SSH] Authenticating as build/******.
java.io.IOException: Password authentication failed.
at com.trilead.ssh2.auth.AuthenticationManager.authenticatePassword(AuthenticationManager.java:319)
at com.trilead.ssh2.Connection.authenticateWithPassword(Connection.java:314)
at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:565)
at hudson.plugins.sshslaves.SSHLauncher.launch(SSHLauncher.java:179)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:199)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Authentication method password not supported by the server at this stage.
at com.trilead.ssh2.auth.AuthenticationManager.authenticatePassword(AuthenticationManager.java:289)
... 9 more
[12/15/11 13:23:12] [SSH] Connection closed.


The host has SLES OS and I was wondering what's going wrong, even though I gave correct password.

FIX: Change the sshd configuration
   In /etc/sshd_config (or /etc/ssh/sshd_config) file
  # To disable tunneled clear text passwords, change to no here!
PasswordAuthentication no
If set to no, Hudson's SSH client won't be able to connect using password authentication. If you comment this line out (which I believe is the OpenSSH default), or set it to yes, then Hudson can connect just fine.


Wednesday, September 1, 2010

Error while starting hudson as a service on Windows

Error

Installing a service
[hudson] $ C:\hudson\hudson.exe install
WMI.WmiException: StatusServiceExists
   at WMI.WmiRoot.BaseHandler.CheckError(ManagementBaseObject result)
   at WMI.WmiRoot.ClassHandler.Invoke(Object proxy, MethodInfo method, Object[] args)
   at WMI.Win32ServicesProxy.Create(String , String , String , ServiceType , ErrorControl , StartMode , Boolean , String[] )
   at winsw.WrapperService.Run(String[] args)
   at winsw.WrapperService.Main(String[] args)