Working with perforce streams - command line

Streams intro

Creating a stream depot

 - Command:    p4 depot <stream-depot-name>

         Ex: # p4 depot DPM

 - Change the type to stream from local

         Type:   stream

Create a new main-line stream

 - Ex: #  p4 stream -t mainline //DPM/MAIN

 - Retain the defaults

Switch your existing workspace to the new stream

 # p4 workspace -f -s -S //DPM/MAIN

Branch files from another depot location

# p4 copy -v //depot/DPM/RTJC/... //DPM/MAIN/...

# p4 submit -d "Creating DPM mainline"

Now the //DPM/MAIN stream is ready to use. To work on it, switch your workspace with the command 

       # p4 workspace -f -s -S //DPM/MAIN

      # p4 sync

      # p4 edit

      # p4 submit

Branching a Development stream from mainline

# p4 stream -t development -P //DPM/MAIN //DPM/DEV

Switch your workspace to work on this new stream

# p4 workspace -s -S //DPM/DEV

Populate the DEV stream by branching files from MAIN

# p4 merge -S //DPM/DEV -r

# p4 submit -d "Branching from mainline"

# p4 resolve -am

# p4 submit -c 7

Merging files from mainline

Now there are changes in mainline, which needs to be merge down to development stream.

   p4 workspace -s -S //DPM/MAIN

   p4 sync //DPM/MAIN/startbuild

   p4 edit //DPM/MAIN/startbuild

   p4 submit -d "bugfix"

 Now switch back your workspace to DEV stream

    p4 workspace -s -S //DPM/DEV

    # p4 merge -S //DPM/DEV -r

        //DPM/DEV/startbuild#1 - integrate from //DPM/MAIN/startbuild#2

    # p4 resolve

    # p4 submit -d "Merged latest changes"

Promoting changes in Dev stream to main

Now we made changes in Dev stream, which needs copy up to main.

   p4 edit //DPM/DEV/startbuild

   p4 add info.txt

   p4 submit -d "Dev changes"

Copy-up only after merging down. So lets merge down again

  # p4 merge -S //DPM/DEV -r

    All revision(s) already integrated.

Now switch back your workspace to main

  # p4 workspace -s -S //DPM/MAIN

  # p4 update     --> To synchronize the workspace with its view of the depot

Copy the content from DEV stream to Main stream

 # p4 copy -S //DPM/DEV

     //DPM/MAIN/info.txt#1 - branch/sync from //DPM/DEV/info.txt#1

     //DPM/MAIN/startbuild#2 - sync/integrate from //DPM/DEV/startbuild#3

 #p4 submit -d "New changes from Dev stream"

Stream types

1) mainline

    Sample spec:

    ----------------------------------------------------------------------

    Stream: //DPM/MAIN

    Owner:  guruss1

    Name:   MAIN

    Parent: none

    Type:   mainline

    Options:        allsubmit unlocked notoparent nofromparent

    Paths:

        share ...

    ----------------------------------------------------------------------

2) development

  

  Sample spec:

    ----------------------------------------------------------------------

   Stream: //DPM/DEV

   Owner:  guruss1

   Name:   DEV

   Parent: //DPM/MAIN

   Type:   development

   Options:        allsubmit unlocked toparent fromparent

   Paths:

        share ...

     ----------------------------------------------------------------------

3) release

 Sample Spec:

 ----------------------------------------------------------------------

  Stream: //DPM/REL-3.5

  Owner:  guruss1

  Name:   REL-3.5

  Parent: //DPM/MAIN

  Type:   release

  Options:        allsubmit unlocked toparent nofromparent

  Paths:

        share ...

 ----------------------------------------------------------------------

Creating a Release stream from mainline

#p4 stream -t release -P //DPM/MAIN //DPM/REL-3.5

Copy up or merge to release streams - Stopping the flow

Copy up or merge to release streams from parent branches are restricted by default.

# p4 merge -S //DPM/REL-3.5 -r

  Stream //DPM/REL-3.5 not configured to 'copy' or 'merge' changes in this direction.

#$ p4 copy -S //DPM/REL-3.5 -r

   Stream //DPM/REL-3.5 not configured to 'copy' or 'merge' changes in this direction.

Flushing release stream with main content

-Switch workspace to release stream

   # p4 workspace -s -S //DPM/REL-3.5 

   # p4 sync

- Forcefully copy up the content from main to release

  # p4 copy -F -S //DPM/REL-3.5 -r

- p4 submit -d "Creating REL-3.5 from main"

Stream spec options

Default options for the stream types

  main:               allsubmit unlocked notoparent nofromparent

  development:    allsubmit unlocked toparent fromparent

  release:             allsubmit unlocked toparent nofromparent

Understanding these options

 - The “fromparent” and “toparent” options toggle whether parent-to-stream and stream-toparent flow are expected at all. The presence of 'nofromparent' in release stream avoid it from accepting changes from parent.

 - “allsubmit” -> “ownersubmit”.  Only owner submit changes to that stream. Content may be pulled from an “ownersubmit” stream into other streams.

 - “locked” - it prevents other users from modifying the stream spec

Understanding stream view Paths

1) Paths:

        share ...

    It means - Share changes with other streams. These paths appear in both client views and branch views, and are mapped to the stream. These files can be synced, submitted, and integrated.

2) Isolate

    Isolate changes from other streams. These paths appear only in client views and are mapped to the stream. That means you can sync and submit the files in these paths, but you can’t integrate them. This is useful for built binaries, generated docs, and other content that you want checked in but not branched or merged.

3) Import 

   Import a path from elsewhere. These paths appear only in client views and are mapped per the parent stream’s view (the default), or to another depot location.You can sync these files, but because they’re not mapped to the stream itself, you can’t submit them. Nor can you integrate them. All they do is mirror other paths.

4) Exclude

  Exclude a path from both client views and branch views. This keeps unneeded content out of workspaces and streams entirely.

Perforce and cygwin

Are you a command-line freak ? Do you want your automated shell scripts to run on Windows ? Do you wish to work with Perforce commands on Cygwin?

You would have experienced the difficulty of running perforce commands on cygwin. The problems which occur due to cygwin's inability to convert Windows paths to it's own /cygdrive/c equivalent.
Example:
  $ p4 have pom.xml
       Path '/cygdrive/c/p4_workspaces/guruss1_p4server2_workspace/depot/On-Prem/AA/7.5.0.0/application/adaptive-authentication\pom.xml' is not under client's root 'C:\p4_workspaces\guruss1_p4server2_workspace'.

Solutions:

1) Use perforce cygwin client: 

    Perforce provide special 'p4.exe' which resolves all these issue. We just need to download it and make it available from default path.

    Where to download ?

       http://filehost.perforce.com/perforce/ 

       The available latest version which I found is     http://filehost.perforce.com/perforce/r12.1/bin.cygwinx86/ 

   The last release is made on May/2012 and hence it got the support for perforce's latest features like Streams, SSL , etc.

   But perforce announced the discontinuity of support for Cygwin p4 client from May/2013 

   Refer http://forums.perforce.com/index.php?/topic/1551-cygwin-client-support/ 

2) Add below function to your cygwin's ~/.bashrc

    function p4() {

        export PWD=`cygpath -wa .`

        /cygdrive/c/Program\ Files/Perforce/p4.exe $@

    }

    This will take care of all path related issues and we can use the same perforce client delivered along with P4V.

3) Add AltRoots: field to your workspace ( or client) spec

     Example:

       AltRoots: 

        /cygdrive/c/p4_workspaces/guruss1_p4server2_workspace   

    This doesn't solve all the problems as I have observed. It works for 'p4 sync', but it didn't for 'p4 have'

How to find source changelist number used to create a perforce branch?

All perforce admins will mostly get this query "Hey from what changelist number branch //branch/B is created from branch //branch/A?"
There is no simple single command from Perforce to answer this question. But still  you can figure it out with below method. Though it is not 100% perfect answer, I can bet it will give 99.99% right one.

Scenario:
 Branch //release/dpm-3.1/KMS/... is created from //dev/rkm/KMS/... a year ago and we didn't preserve the source branch (i.e. //dev/rkm/KMS/...) changelist number used to create new //release branch.
 Usually the standard practice is to record this changelist number in the description of new branch changelist while creating it.
 Anyways, if you have missed it, no worries, still you can figure it out.

Solution:
Step 1: Find the first changelist number which got created when the new branch //release/dpm-3.1/KMS/... took birth. You can use below command to find it.
   $ p4 changes //release/dpm-3.1/KMS/...|tail -1
  Change 1270598 on 2011/11/22 by guruss1@sguru_VWINRSA2-46 'Creating //release/dpm-3.1/KMS '

  Note down this number 1270598

Step 2: Find the ancestral history of new branch //release/dpm-3.1/KMS/... using 'p4 changes -i' command and find out what was the changelist number before the birth of branch (i.e changelist before 1270598). Note that -i option will traverse behind and tell what all parent branch changelists are integrated to this new branch.
You can use below command to find ancestral history. Here I'm limiting history to 3 changelist numbers before this branch took birth.
  $  p4 changes -i //release/dpm-3.1/KMS/...|grep -A3 '1270598'
Change 1270598 on 2011/11/22 by guruss1@sguru_VWINRSA2-46 'Creating //release/dpm-3.1/KMS '
Change 1254271 on 2011/09/20 by sguru@sguru_VWINRSA2-46 'Adding thirdpartylicenses.pdf f'
Change 1254094 on 2011/09/19 by pasuns@Sreeekanth_RKM 'coverage reduced to 70% to make'
Change 1253772 on 2011/09/19 by pasuns@Sreeekanth_RKM 'Fix for KMSRV-1798: Addressing '

From this output, I got to know that, changelist number 1254271, which is from the parent branch  //dev/rkm/KMS/... is the last changelist number integrated and hence this is the changelist used to create this branch. 

Firewall configuration. How to open a port on Linux?

This summary is not available. Please click here to view the post.

Running perforce p4d as service on RHEL

Perforce is not providing standard script to run the perforce server p4d as service on Linux systems. Different people configured it in different way. Here is my way of running it as service on RHEL machines.

• Create an account 'perforce' in your machine
• Download p4d and place it in designated directory as defined P4ROOT. Provide exec permission for it. And also change the owner to 'perforce' account.

         In my case P4ROOT=/home/perforce/server

• Create a start-up script 'p4d' at /etc/init.d

cat /etc/init.d/p4d
#!/bin/sh
# description: This is a daemon which starts perforce server on reboot
PATH=/sbin:/bin

test -f /home/perforce/server/p4d || exit 0
export P4ROOT=/home/perforce/server
export P4PORT=1818
RUN_AS_USER=perforce

case "$1" in
start)
  su -m $RUN_AS_USER -c "/home/perforce/server/p4d -r $P4ROOT -J /home/perforce/logs/journal -L /home/perforce/logs/p4err -p tcp:$P4PORT &"
  ;;
stop)
  /usr/local/bin/p4 admin stop
  ;;
*)
  echo "Usage: /etc/init.d/p4proxy {start|stop}"
  exit 1
esac

exit 0

Here note that we are not running p4d as root, instead using the account 'perforce'

• Add a new service for management through chkconfig

          chkconfig --add p4d

• Configure the run levels on which it needs to be on

          chkconfig --levels 345 p4d on

        It creates soft-links like

           ls -ltr /etc/rc3.d/S29p4d

        lrwxrwxrwx 1 root root 13 Sep  4 16:36 /etc/rc3.d/S29p4d -> ../init.d/p4d

• Verify it by running commands

            service p4d start

            service p4d stop

Perforce sync is slow - Checklist for debugging

Is your Perforce performance is slow while doing sync or some other operations ?


1) Check you client spec’s view (if the repository is large) and if your client has a view of the entire repository, performance will be impacted (or you may have exceeded a maxRowCount error)
2) Modify your client spec (or create a new one) with a very narrow view (for testing, maybe down a single folder)
3) Do you have a firewall setup and preventing this kind of activity?
4) Can you try sync from another machine?
5)Try and play with "p4 ping" by progressively increasing the  message count, receive and transmit lengths (one at a time). If you hit the same wall repeatedly, then this could very well be indicative of some   throttling taking place somewhere
6) p4 -z tag info
7)  p4 -vN sync //depot/path/to/file      where N = 1,2,..9
8)  Network problems  http://kb.perforce.com/article/40/isolating-network-problems

 

Learning the use of "p4 duplicate" command

Another way is to deep rename it:

p4 duplicate //depot/bad_branch_name/... //depot/correct_branch_name/...

Now at that point take a look at the revision graph of some of the files. You'll see that even the integration history is duplicated. This is a command that effectively changes the past.
then:

p4 obliterate (-y) //depot/bad_branch_name/...

However, depending on where the librarian files actually are (check using p4 fstat -Oc //depot/bad_branch_name/... and look for the lbrFile value) you may want to relocate them to the renamed branch before obliterating if they are not lazy copies. Using p4 snap, so for example p4 snap //depot/bad_branch_name/... //depot/correct_branch_name/...

How to create perforce user non-interactively

I was trying to create perforce user account using “p4 user -i” option. I wanted to bulk rename perforce login-ID's due to change in our organization policy.

Here is the command.
p4 user -i -f < tempfile.txt

Where typically a tempfile.txt will be of the form
----------------------------------------------------------------------------------------------
# cat user-template.txt
# A Perforce User Specification.
#
#  User:        The user's user name.
#  Type:        Either 'service' or 'standard'. Default: 'standard'. Read only.
#  Email:       The user's email address; for email review.
#  Update:      The date this specification was last modified.
#  Access:      The date this user was last active.  Read only.
#  FullName:    The user's real name.
#  JobView:     Selects jobs for inclusion during changelist creation.
#  Password:    If set, user must have matching $P4PASSWD on client.
#  Reviews:     Listing of depot files to be reviewed by user.

User:   guruss1

Email:  siddesh.gurusiddappa@emc.com

Update: 2011/10/18 05:04:00

Access: 2011/10/18 06:17:32

FullName:       siddesh

----------------------------------------------------------------------------------------------



In context of the perl script:
open (WRITE, ">", "tempfile.txt") or die "Can't open temp file for writing.\n";
print WRITE "User: $uName\n\n";
print WRITE "Email: $email\n\n";
print WRITE "Update: $date\n\n";
print WRITE "FullName: $fName $lName - $prefix\n\n";
close WRITE;
system( "p4 user -i -f < tempfile.txt");

Calculating LOC(Line of changes) between releases on Perforce

0509-036 Cannot load program p4 because of the following errors

Here is the full description of error ............
bash-3.00# p4 info
exec(): 0509-036 Cannot load program p4 because of the following errors:
        0509-111   The s_scnptr or s_size field of the
                  .loader section header is invalid.
        0509-194 Examine file headers with the 'dump -ohv' command.

I got this error when I invoke perforce client (p4) from an IBM AIX 5.2 machine.

Whats going wrong here?
We installed p4 client of IBM AIX 5.3 on IBM AIX 5.2 machine

Resolution: Install proper p4 client for AIX 5.2 machine, which is available in Perforce archive
Direct link: http://www.perforce.com/downloads/perforce/r05.1/bin.aix43/p4

or
http://www.perforce.com/perforce/downloads/unixlist.html   browse to "IBM AIX 4.3 for PPC"

How to set a login timeout for a perforce user?

Perforce supports two methods of authentication: password-based and ticket-based.

How password-based authentication works?
Password-based authentication is stateless; once a password is correctly set, access is granted for indefinite time periods.

How ticket-based authentication works
Ticket-based authentication is based on time-limited tickets that enable users to connect to Perforce servers. Tickets are stored in the file specified by the P4TICKETS environment variable. If this variable is not set, tickets are stored in %USERPROFILE%\p4tickets.txt on Windows, and in $HOME/.p4tickets on UNIX and other operating systems. Tickets are managed automatically by 2004.2 and later Perforce client programs.

Tickets have a finite lifespan, after which they cease to be valid. By default, tickets are valid for 12 hours (43200 seconds). To set different ticket lifespans for groups of users, edit the Timeout: field in the p4 group form for each group. The timeout value for a user in multiple groups is the largest timeout value (including unlimited, but ignoring unset) for all groups of which a user is a member. To create a ticket that does not expire, set the Timeout: field to unlimited.

Although tickets are not passwords, Perforce servers accept valid tickets wherever users can specify Perforce passwords. This behavior provides the security advantages of ticket-based authentication with the ease of scripting afforded by password authentication.

Logging in to Perforce
To use ticket-based authentication, get a ticket by logging in with the p4 login command:

p4 login

You are prompted for your password, and a ticket is created for you in your ticket file. You can extend your ticket's lifespan by calling p4 login while already logged in. If you run p4 login while logged in, your ticket's lifespan is extended by 1/3 of its initial timeout setting, subject to a maximum of your initial timeout setting.

By default, Perforce tickets are valid for your IP address only. If you have a shared home directory that is used on more than one machine, you can log in to Perforce from both machines by using the command:

p4 login -a

to create a ticket in your home directory that is valid from all IP addresses.


Determining ticket status
To see if your current ticket (that is, for your IP address, user name, and P4PORT setting) is still valid, use the command:

p4 login -s

If your ticket is valid, the length of time for which it will remain valid is displayed.

To display all tickets you currently have, use the command:

p4 tickets

The contents of your ticket file are displayed.


Refer P4 Sys admin Guide for details

Perforce Remote depot creation

The perforce remote depot allows to access files in depot created in another server instance. For example if you are connected to a server listening to port 1666 and want to access contents of a depot created in server listening at port 1667, remote depot assists in doing this. You can sync, diff and integrate remote depot with local depot.

Example:
Server1=1666 : Branch-name=//user/sguru/...
Server2=1888 : Remote-depot=//remote-sguru/... (this is the new depot which will be created)
I need to make //user/sguru/... in server1 as remote depot in server2

Step1: Create a new depot on Server2 and make following changes in depot spec
Depot: remote-sguru
Type: remote
Address: server1:1666
Map: //users/sguru/...

Step2: Open protection table on Server1 and add the following line
read user remote //users/sguru/…

Step3: Grant read access to //remote-sguru/... depot only to your build or integration managers. Open protection table on Server2 and add the following
list user * * -//remote-sguru/...
read user p4admin * //remote-sguru/...
We are avoiding rest of users other than admin accessing directly remote depot (server1) through //remote-sguru/... to minimize network traffic. Instead we create another branch as described in step4 and allow other users to access remote depot code from it.

Step4: Integrate //remote-sguru/... to location and allow rest of the users to access remote data from here. This will reduce network traffic. On server2 do following
p4 integ //remote-sguru/... //users/remote-sguru/...
Add following line to protect table
read user * * //users/remote-sguru/...

Perforce spec depot

No need to manage manually perforce specs like client spec, job spec, label spec, branch spec, protection table, etc. Perforce (from 2005.1 release) provides a special depot called specs depot which take care of managing these specs by automatically creating a new version under //spec for each spec modifications.
How to create spec depot?
- p4 depot spec (or any other name U wish)
- In type field change it to spec (by default it comes local)
- Save it. It creates //spec depot. From now on any modifications to specs are stored under //spec depot. For example client spec changes will be stored in //spec/client/client-name.p4s

How to populate spec depot with change details for specs already created?
Run command "p4 admin updatespecdepot -a" (This command available from 2007.3 release).
If you want to restrict population to specific type of spec, then modified command will be
p4 admin updatespecdepot -s spec
Ex: p4 admin updatespecdepot -s client

Perforce resources

Perforce official Blog/
Perforce twitter
Perforce knowledge base
Perforce Website
Perforce Mail list perforce-user@perforce.com
Perforce @ CM crossroads

Perforce branch locking

You can lock Perforce branch,depot or any other folder by using either
1) Protection table
2) Triggers

Using protection table:
You can lock a branch by just creating a new rule in Protection table. For example here, we are locking a branch called //dev/envision/esi/... by just providing only read permission to group xxx

read group xxx * -//dev/envision/esi/...


Here is a complex example.
The below protection table locks the branch //release/envision4.1sp1/... for every one else, except the user xxxats2. It locks for all the concerned groups release-xxx-4.0, xxx, yyy-group, yyy-all

                =write group release-xxx-4.0 * -//release/envision4.1sp1/...

                =write group xxx * -//release/envision4.1sp1/...

                =write group yyy-group * -//release/envision4.1sp1/...

                =write group yyy-all * -//release/envision4.1sp1/...

                write user xxxats2 * //release/envision4.1sp1/...

                =write group ddds * -//user_information/enVision/4.1_SP1/output/...

                =write group uuuu-info * -//user_information/enVision/4.1_SP1/output/...

                =write group yyy-all * -//user_information/enVision/4.1_SP1/output/...

                write user xxxats2 * //user_information/enVision/4.1_SP1/output/...

Points to remember:

• The protection table order works from bottom to top. Hence place new locking rules below the existing rules in order for it be in effect.
• =write : means, provide all other privileges ( like read, open), except write
• Use "p4 protects //release/envision4.1SP1/..." command to find out which all groups/users already have write privileges. Remove write privileges from all the concerned groups

Using triggers:
You can also lock a branch by writing a changelist submission trigger of type change-submit. Here the trigger logic work something like, when a changelist comes for submission, check for files under branch which you want to lock and also check for some .lock file created in a location accessible to server. If .lock file is present then cancel change submission.
Cons: This will definitely slowdown the CL submission time.

Perforce - can't edit exclusive file already opened

In perforce, whenever a binary file like doc, xls or ppt files are checked out, it is opened in exclusive lock mode. So no other person can check-out that file until the current person who checked-out submit(check-in) it. But there is a way to check-out binary files in these scenario

Check out your binary file as shown below
p4 edit -t binary //knowledgebase/ctweb/main/releases/AXM/Server/Moneypenny/Dev/moneypenny_code_review.xls

-t binary option will allow you to check-out binary file even it is locked. After your modification, you can check it back in without any issues

Perforce commands FAQ

1) How to delete a Perforce client?
p4 client -d

2) How to copy named template's view to current client?
p4 client -t

3) What is the use of P4CONFIG environment variable?
You can define Perforce environment variables in this configuration file. Place this file in your client root or even top-level directory. Perforce will fetch values from this configuration file.
For Ex: setenv P4CONFIG .p4config
cd $P4ROOT
vi .p4config
P4PORT=localhost:1666
P4CLIENT=client1

4) How to retrieve a file revision from Perforce depot?
p4 print -q //depot/source/proj1/test.c#5 > test.c

5) How to add the entire directory to Perforce?
find . -type f|p4 -x- add
find . -type l|p4 -x- add -t symlink

6) How to check who all working(opened) on a given file?
p4 opened -a

7) How to list opened files that are different than the revision in the depot or missing?
p4 diff -sa

8) Listing unopened files that are missing on the client
p4 diff -sd

9) Listing unopened files that are different from the revision in the depot
p4 diff -se

10) Listing opened files that are same as the revision in the depot
p4 diff -sr

11) How to revert all files opened for edit but have unchanged content?
p4 revert -a

12) Locking an opened file against submission
p4 lock file-name
To unlock
p4 unlock file-name

13) How to list the differences between the 2 branches?
p4 diff -q //depot/main/jam/... //depot/rel1/jam/...

14) Deleting and re-adding files
% p4 delete file.txt
//depot/src/file.txt#33 - opened for delete
% p4 submit
Delete //depot/src/file.txt#34
Change 145 submitted
% p4 sync file.txt#33
% p4 add file.txt
%p4 submit
Add //depot/src/file.txt#35
Change 146 submitted

Beginning Perforce

• What is Perforce?

Perforce means "By necessity; by force of circumstance". Now every software organization necessarily need a Source control tool, perforce is one among them. Perforce provides a force to manage the source code efficiently and effectively.

• Want to play with Perforce? Download

Download a free 2 user Perforce server from http://www.perforce.com/perforce/loadprog.html. To play around, a Perforce server "p4d" and a Perforce client "p4" is required on Unix platforms or you can download p4win for windows, which is GUI based includes P4, P4D and P4P.
P4P is perforce proxy. P4V is mainly used as GUI client in UNIX machines, but it can also be used in Windows.

• How to start a Perforce Server?

In Unix:
mkdir $HOME/p4bin. Put P4, P4d and P4V to here and add this directory to your path
mkdir $HOME/p4server
mkdir $HOME/p4workspace
Make the programs in $HOME/p4bin executable

Starting the Perforce server:
p4d -d -p 1666 -r $HOME/p4server
where -d suggests to run server as daemon
-p tells port number
-r set server root directory

In Windows:
Install p4win
Start Perforce server p4d.exe as
./p4d.exe -d -q -p 1777 -r . -L ./log &
where -q suggests quiet (suppress start up messages)
-L sets error Log

• Terminologies used in Perforce

Depot: Repository for source files and Metadata.
Client: Your private workspace to download files from Depot and to play with it
Sync: Get code from depot to client
Edit: Check out
Submit: Check in

• Creating a Perforce client

Perforce client is your own private work space to sync the depot code