Committing content to RTC SCM with the SDK

Now comes the fun part: committing changes to Rational Team Concert's SCM. We're going to write a program that commits new content to a file in RTC without loading it first. This isn't possible in the Eclipse, Visual Studio, or command line interfaces.

Note: This post uses internal API. It may change at any moment, stranding you on a specific version of RTC. 

You can download a zip of the Eclipse project, or load it from JazzHub.

Relevant Architecture

As you know, RTC SCM has repository workspaces. Workspaces record the directory structure of source code in the repository. We refer to files and folders in the workspace as items. Whenever a user changes an item, a change set is created that moves the item from a before state to an after state. 

If the user modifies the content of a file, the before state in the change set has the old text while the after state has the new bytes. 

Internally, items have state ids that are used to uniquely identify the before and after states.

Change sets are either active or complete. An active change set can be given new after states, while a complete change set cannot have new after states associated with it.

To avoid ambiguity, an item can only appear in one active change set per workspace. In other words, if I want to commit a new change to some item that is already in an active change set, I must commit to the existing change set.

Object Model

In this example we're going to be diving deeper into the guts of the Jazz platform than usual, so you'll need to know a little about how it works. Here's a quick sketch.

The RTC server stores objects. Every object has a unique item ID as well as a unique state ID. When one object references another, it does so with a handle. The handle consists of the item ID and, if the object cares about a specific version, the state ID. Implicitly both the object and handle have a type. 

To minimize bandwidth consumption, the RTC server and client usually exchange handles. But eventually they need to exchange real data. When that happens, the client will fetch the full object. In our example, when the client wants to learn more about a change set, it will fetch that change set. 

Fetched objects are immutable. The client cannot change them without getting a working copy first. Fields on the working copy may be set as appropriate, then the client saves it back to the RTC server. When the client modifies the object representing a file, it creates a working copy of that file. 

We'll see more and more of handles, fetches, and working copies as we explore this sample.

Pseudocode 

Our uploader will perform the following steps:

1. Find the repository workspace and component named in the arguments. 
2. Find the file item in the workspace. 
3. Upload the file content to the repository.
4. Choose a change set to record the modification. 
5. Tell the repository to set the after state of our change set to be the uploaded content. 

You'll notice that step three is a little weird: we upload content and then create the change set, rather than doing it in one step. Welcome to the land of implementation details. 

API and SDKs

This post uses internal API. It may change at any moment, meaning that any code you write will be tied to a specific version of RTC. So don't get upset if that happens. In practice, this code is pretty stable, so it probably won't happen any time soon, but...

We're going to be committing from the client, so we need the following classes:

• IWorkspaceConnection - the logical representation of a repository workspace available in the client side SDK. It provides handy caches and methods simplifying workspace access. 
• IFileContentManager - provides read/write access to file content stored in the RTC SCM. (this is not supported API)
• IFileItem - a file in RTC SCM. (this is not supported API)
• IConfigurationOp - describes a change to an item in a repository workspace. The repository uses these to build change sets. 

We're using a 4.0.3 version of the server and SDK. Nothing here is specific to that version, so it should work with newer and older versions. Notice I say should.

Implementation

Our example program will take a number of arguments, including:

1. the workspace to commit to
2. the component to commit to
3. the path in the repository workspace
4. the path to the new file content

I'm not going to cover resolving the workspace and component names, since we've covered that before. Instead, we'll start on line 95, where we resolve the remote path.

Resolving the Repository Path

In order to commit changes to an existing item, we need to find the item's ID. Humans, the frail beings that we are, refer to files by path. But RTC doesn't care about paths and refers to files by their item ID. So we need to find the IFileItem for the path the user gave us on the command line. 

We do that in findRepositoryPath() by splitting the user's path into segments, then asking the IWorkspaceConnection for the directory structure of the component. The structure is represented by an IConfiguration, which allows us to query the item at the user's path:

private IFileItemHandle findRepositoryPath(IWorkspaceConnection wsConn, IComponent comp, String remotePath) throws TeamRepositoryException {
 String[] path = remotePath.split("(/|\\\\)");
 
 IConfiguration config = wsConn.configuration(comp);  
 IVersionableHandle itemAtPath = config.resolvePath(comp.getRootFolder(), path, null);
 
 if (itemAtPath == null) {
  throw new RuntimeException("Could not resolve path " + Arrays.asList(path));
 }
  
 if (itemAtPath instanceof IFileItemHandle) {
  return (IFileItemHandle) itemAtPath;
 }
  
 System.err.println("We only like files, not " + itemAtPath.getClass().getSimpleName());
 throw new RuntimeException();
}

Line 143 performs the path query. The first argument is the root folder, which the the root of the component, and the second argument is the path specified by the user. The third argument is a IProgressMonitor - we don't care about those for this example.

The return value is a handle to an IFileItem, which carries the item type, the item id, and state id. Our example is limited to dealing with files, so we use an instanceof check on line 149 to ensure that it's a file. Repository workspaces can also hold folders (represented by IFolder) and symbolic links (represented by ISymbolicLink) but those would complicate the example, so we're ignoring them.

Uploading New Content

We upload content with the IFileContentManager. In RTC, content knows a few things about itself, namely the encoding, line delimiter (for text files), and an optional previous version. Since this is an example, we hardcode reasonable values: 

// Upload content
System.out.println("Uploading content of " + contentSource.getAbsolutePath());
IFileContentManager contentManager = FileSystemCore.getContentManager(repo);

IFileContent content = contentManager.storeContent("UTF-8", FileLineDelimiter.LINE_DELIMITER_NONE, new FileInputStreamProvider(contentSource), null, null);

The content is read multiple times during the upload, so we can't pass around an InputStream, since the first close() would render it useless. Instead, the caller passes in a factory that allows the stream to be repeatedly opened.

The multiple reads are due to a couple of implementation decisions: 

1. Our content transport layer is unadulterated HTTP 1.0. In order to use Connection: Keep-Alive, we need to know the length of the stream before we upload it. We don't use content chunking. I don't know why. 
2. We can't use the file size to determine the length, because we normalize *nix/DOS line endings on upload. Our only option is to walk the file to determine the length. 

But callers don't need to know that. Instead, they just have to subclass AbstractVersionedContentManagerInputStreamProvider and define reasonable methods. The FileInputStreamProvider is probably the minimal possible implementation. You can see it on line 49 of TrivialCommit.java

The upload occurs as a single call and returns an IFileContent object. The content object is a client-side references that is used to uniquely identify file text in the repository. 

Creating the After State

There are two arcane steps we must take before creating the 'after' state for our change set: we need to inflate the IFileItemHandle from findRepositoryPath() into a full item, and then get a working copy of that full item.

IFileItem fileItem = (IFileItem) wsConn.configuration(comp).fetchCompleteItem(fileHandle, null);
fileItem = (IFileItem) fileItem.getWorkingCopy();

As we said above: items are passed between the RTC server and client as a handle consisting of the item id, the state id, and the item type. Handles can be considered cross-network pointers: they're a lightweight representation that allows programmers to refer to the item without worrying about its properties. In this case, we care about the IFileItem itself, so we get the full representation on line 106 by fetching it with IConfiguration.fetchCompleteItem(). 

Full items are immutable, so we have to ask the item for a working copy of itself on line 107 before we can change it. Once we have the working copy, we make changes to that. 

The handle/full-item/working-copy trichotomy gives us some powerful tools. 

• We can pass handles across the network quickly, only fetching the full version of the ones we care about. 
• We can batch fetching handles, to minimize the number of network operations.
• Immutable full items allow long-lived portions of RTC to assume that the properties of the item haven't changed. 
• Eventing when working copies are saved allow the RTC UI to update appropriately. 

Recording the Change

Our changes to the IFileItem are minor: we just update the content and modification date. The other fields keep the previous values.

fileItem.setContent(content);
fileItem.setFileTimestamp(new Date());

The working copy of the IFileItem is the 'after' state of the file change we want to make. To get the 'after' state into a change set we create an IConfigurationOp containing the working copy. The configuration op is used to inform the repository that a change set should be updated.

// Save the change into a change set
IConfigurationOpFactory opFactory = wsConn.configurationOpFactory();
ISaveOp saveOp = opFactory.save(fileItem);

We have IConfigurationOps because we want a common way of talking about item modifications. Aside from the Milquetoast SaveOp, we have ops that merge conflicts, delete items, or remove them from change sets.

We're almost done. We just need to find a change set to record the after state.

Finding the Change Set

In a truly trivial example, we would create a new change set and commit into that. But there's a restriction on commit: an item can only appear in one active change set per workspace. If we try committing it to another change set, the RTC repository will throw an exception. This use case is fairly common, so we'll handle it in selectChangeSetModifying().

private IChangeSetHandle selectChangeSetModifying(IWorkspaceConnection wsConn, IComponent comp, IFileItemHandle fileHandle) throws TeamRepositoryException {
 List<IChangeSetHandle> activeChangeSets = wsConn.activeChangeSets(comp);
  
 @SuppressWarnings("unchecked")
 List<IChangeSet> changes = wsConn.teamRepository().itemManager().fetchCompleteItems(activeChangeSets, IItemManager.DEFAULT, null);
 
 for (IChangeSet cs : changes) {
  for (IChange change : (List<IChange>)cs.changes()) {
   if (fileHandle.sameItemId(change.item())) {
    return cs;
   }
  }
 }
 
 return wsConn.createChangeSet(comp, null);
}

To avoid the exception, we walk the set of active change sets and look for one with a change to our IFileItem. The IWorkspaceConnection knows the list of active change sets, but only records them as handles, so we convert the IChangeSetHandles into full items with the IItemManager on line 240.

The IChangeSet expresses changes as IChange objects. Each of those modifies a single item, so we walk those to see if our IFileItem is already being modified.. We know that only one change set may modify an item at a time, so we're safe to return when we find the first match on line 245.

There's always the possibility that there aren't any active change sets modifying our IFileItem, so line 250 creates a new IChangeSet if necessary.

Committing the Change

We have all of the parts we need: an ISaveOp describing the after state of our item, a change set to record the change, and a workspace containing the change set. Our commit is almost anticlimactic: 

// Save the change set
System.out.println("Committing");
wsConn.commit(cs, ops, null);

After the commit is over, we change the comment on the change set and complete it. We do that to make our change set easier to identify. In production code we would (probably) leave the change set open and we certainly wouldn't create such a pointless comment, since the completion time of the change set already has that information. 

Running the Program

The program takes a whopping seven arguments: the first three are the repository URI, user name and password. The next three are the repository workspace name, component name, and path of the item to save to. The last argument names the file that contains the bytes we want to commit.

Let's put together a simple example:

1. Start your repository and connect to it with the RTC Eclipse client.
2. Create a trivial repository workspace that has a few directories then open it in the Repository Files view. Mine looks like:
   
   The file text.txt is the target of our commit. 
3. We'll use the eclipse 'Trivial Commit' launch. You need to modify the paths and arguments before you can use it. You can do that by opening the Run menu and executing the Run Configurations gesture. Edit the 'Trivial Commit' launch and modify the arguments as appropriate:
   
   The arguments are: repository URI, username, password, workspace name, component name, path of the item to commit in the repository workspace, and the path of the content to write to the remote workspace. 
4. My initial content for text.txt is an empty file, while the content of /tmp/sample.txt is 'hello commit world'. To verify your content, open the target file from the Repository Files view in Eclipse. 
5. Run the launch. It should chug for a few moments before performing the commit. 
6. Refresh the Repository Files view and open text.txt.
7. Et voilà!
   
   
   The file's history contains a change set with our automatically created comment:
   

Now you have a working commit using the RTC SDK. The sample demonstrates how to find an item in a repository workspace, traverse active change sets to one that modifies a specific item, and add a change to a change set. Along the way you've learned a little more about RTC's item model. 

As always, comments are welcome, and the source code is available on JazzHub.