As the amount of CSS source code grew, it has been split into several modules. To build a CSS product from these, there are two extreme approaches.

Obtaining and compiling the complete source code is time consuming. In practice, you often use an intermediate approach. You obtain the source code for modules that you want to investigate further, for example to fix a bug or to add new functionality. For the remaining modules, you rely on binaries in a remote P2 repository.

Which exact approach you use depends on your situation. You may want to configure a local Hudson/Jenkins instance to perform a complete nightly build for your site, obtaining all the source code and compiling it. Or you may prefer to configure your Eclipse IDE target platform to use remote P2 repositories, then import only the source code for a site specific product and run it in the IDE. Read through the remainder of this chapter to learn about the options and tools, then make your decision.

The complete CSS sources are in a shared GitHub repository using the GIT version control system. The project web site is https://github.com/ControlSystemStudio.

For GIT in general, see http://git-scm.com.

Below is an example for fetching a complete copy of the source code. If you choose to rely on remote binaries for all but the product, you would only perform step 4.x.

# Define base URL
GH=https://github.com/ControlSystemStudio

# 1) Maven bundles
git clone $GH/maven-osgi-bundles

# 2) Third party
git clone $GH/cs-studio-thirdparty

# 3) Diirt
git clone $GH/diirt

# 4) Bulk of CSS sources
git clone $GH/cs-studio

# Depending on your site, you would only fetch one of the following
# examples, or use one that you created for your own purposes

# 5.1) 'common' product
git clone $GH/org.csstudio.product

# 5.2) SNS products
git clone $GH/org.csstudio.sns

# 5.3) NSLS2 product
git clone $GH/org.csstudio.nsls2.product

# 5.4) Your product?
git clone you_would_need_to_know_what_to_get

The master branch of each repository contains the most recent set of sources. For each release, there is a numbered release branch, for example 4.0.x. New functionality and bug fixes are added to the master branch, while the release branches are only updated for essential bug fixes.

To work on the CSS source code, i.e. to be able to submit changes, several steps are necessary:

When you compile CSS sources, building each module results in a P2 repository of artifacts. Compilation of a module typically depends on other previously created modules. For example, compiling the cs-studio sources requires access to P2 repositories for the maven-osgi-bundles and cs-studio-thirdparty artifacts.

When you compile from sources, these P2 repositories will be created in subdirectories repository/target/repository of your source tree. If you prefer to skip local compilation and instead use remote repositories, these are under http://download.controlsystemstudio.org. In the following description of remote repository URLs, note that each URL contains a version number. When working with a relased version, the URL simply contains the version number. For the source code on the git master branch, the URL uses the next to-be-released version number.

The Eclipse IDE as well as each CSS product contain the P2 Director application which can be used to list the content of a repository:

# Use either the Eclipse IDE or a CSS product.
# Both should include the Equinox launcher
# plugin which in turn contains the P2 director
export APP_PLUGINS=/path/to/my/eclipse

DIRECTOR="java \
 -jar $APP_PLUGINS/plugins/org.eclipse.equinox.launcher_*.jar \
 -debug -consolelog \
 -application org.eclipse.equinox.p2.director"

# List content of a remote repository.
$DIRECTOR -list -repository http://download.eclipse.org/releases/luna/

# List content of a local repository.
# Path needs to start with 'file://' and be absolute.
$DIRECTOR -list -repository \
 file://path/to/maven-osgi-bundles/repository/target/repository

Assembling a product typically requires artifacts from maven-osgi-bundles, cs-studio-thirdparty, cs-studio/core, cs-studio/applications and maybe locally created site-specific additions.

A composite P2 repository is simply a list of other P2 repositories. It can list local as well as remote repositories, which is useful because you typically need to use a combined approach. For example, you may want to use remote repositories for maven-osgi-bundles and cs-studio-thirdparty, but prefer locally created artifacts for the rest.

To create a composite repository, create a directory my_comp_repo in the same directory that also contains the previously cloned source modules with the following two files:

<!-- File my_comp_repo/compositeContent.xml -->
<?xml version='1.0' encoding='UTF-8'?>
<?compositeMetadataRepository version='1.0.0'?>
<repository name='Local Composite Repository'
type='org.eclipse.equinox.internal.p2.metadata.repository.CompositeMetadataRepository'
            version='1.0.0'>
  <properties size='1'>
    <property name='p2.atomic.composite.loading' value='false'/>
  </properties>
  <children size="5">
    <child location="http://download.controlsystemstudio.org/maven-osgi-bundles/4.2"/>
    <child location="http://download.controlsystemstudio.org/thirdparty/4.2"/>
    <child location="http://diirt.download.controlsystemstudio.org/diirt/3.1.6"/>
    <child location="../cs-studio/core/p2repo"/>
    <child location="../cs-studio/applications/p2repo"/>
  </children>
</repository>

<!-- File my_comp_repo/compositeArtifacts.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<?compositeArtifactRepository version='1.0.0'?>
<repository name="Local Composite Repository"
type="org.eclipse.equinox.internal.p2.artifact.repository.CompositeArtifactRepository"
            version="1.0.0">
  <properties size='1'>
    <property name='p2.atomic.composite.loading' value='false'/>
  </properties>
  <children size="5">
    <child location="http://download.controlsystemstudio.org/maven-osgi-bundles/4.2"/>
    <child location="http://download.controlsystemstudio.org/thirdparty/4.2"/>
    <child location="http://diirt.download.controlsystemstudio.org/diirt/3.0.1"/>
    <child location="../cs-studio/core/p2repo"/>
    <child location="../cs-studio/applications/p2repo"/>
  </children>
</repository>

In this example, we depend on remote repositories for maven-osgi-bundles cs-studio-thirdparty and diirt, while using locally created artifacts for the rest. For local child locations, entries starting in file:/ refer to the root directory and need to be absolute. The child locations in the example refer to locations relative to the XML files.

For further examples, check the core/p2repo or applications/p2repo which themselves are composite repositories.

The p2.atomic.composite.loading controls how missing child locations are treated. Setting it to true will check if all listed child locations contain valid repositories. This can be used to test the correctnes of the composite repository settings. On the other hand, when we start out fresh and compile the cs-studio/core module, the cs-studio/applications/p2repo will not exist, yet, because it is generated in a later step, so we need to set p2.atomic.composite.loading to false.

Maven is a generic build tool for Java. It uses pom.xml files to define how a piece of software needs to be compiled, and to list its dependencies. Eclipse plugin sources already include META-INF/MANIFEST.MF files for almost the same purpose. Tycho, a support module for Maven, allows Maven to re-use the existing Manifest files to compile Eclipse source code.

From now on, we will refer to the Tycho/Maven Build as simply the Maven Build. To build CSS from sources with Maven, you need

Add the desired version of Java to the JAVA_HOME environment variable as well as the PATH:

export JAVA_HOME=/path/to/the/jdk
export PATH=$JAVA_HOME/bin:$PATH

Verify that Maven is using the desired version of Java. Note that the first Java found on your $PATH may not be what Maven uses because it checks $JAVA_HOME. Verify by checking the output of

mvn -version

When obtaining the Maven Build tools and CSS sources, you may have noted that we fetched neither Tycho nor Eclipse. The beauty of Maven is that it will download all dependencies, including Tycho and Eclipse, and work like a charm as long as it feels like doing so. The horror of Maven is that it always appears to download the internet. When the commands listed in the following instructions are run for the first time, Maven will initially download Tycho. It will place Tycho in a local Maven repository, typically under ~/.m2/repository. Subsequent calls will not require a complete download, but Maven will continue to check remote sites for updates, even when invoking a simple mvn clean command.

Note: If you have used Maven on a previous project, you may be familiar with executing it as mvn install. This will compile the source code and place the results in the local Maven repository. For compiling CSS, this is not recommended. Do not invoke the Maven install command, because this will result in version conflicts between artifacts that Maven then installs in its local repository and the current sources of CSS. Instead, we use mvn verify, and the generated CSS binaries are placed in the previously described local P2 repositories within the source tree.

If you do not further configure Maven, all CSS compilation will default to using remote repositories. For example, if you only obtained the source code for the common CSS product, the following will build it, obtaining all required depdendencies from remote repositories which are configured in the pom.xml files of the product:

# Compile common product
(cd org.csstudio.product; mvn clean verify)

As you develop CSS, you might prefer to download more of the source code so that you can inspect and maybe modify it. You then configure a composite repository that lists your locally created repositories, based on locally edited sources, over the remote repositories. To configure Maven to use your composite repository as well as additional settings, create either a global file $M2_HOME/conf/settings.xml, or in a user-specific file $HOME/.m2/settings.xml. You could also specify the properties listed in this file on the command line, invoking Maven as mvn -Dcsstudio.composite.repo=.., but to assert consistency are best placed in a /.m2/settings.xml file based on the following example:

<!-- Maven settings.xml -->
<settings>
  <profiles>
    <profile>
      <id>my-css-settings</id>
      <properties>
        <!-- Optionally, configure a composite repo.
             By default, the Maven build will fetch all
             depenencies from remote repositories.
             In your composite repository, you can list
             local repositories.
          -->
        <!-- Path to your composite repo -->
        <csstudio.composite.repo>/path/to/my_comp_repo</csstudio.composite.repo>

        <!-- By default, even if you use a composite repo
             that lists local P2 sites,
             Maven will still consult the remote CSS repos.
             Setting this property DISABLES downloads from
             http://download.controlsystemstudio.org,
             so you use _only_ sites listed in your composite repo.
             If you still want to use selected entries from
             http://download.controlsystemstudio.org,
             you can list them in your composite repo.
         -->
        <cs-studio>false</cs-studio>
        <eclipse>false</eclipse>

        <!-- Ignore artifacts in ~/.m2/repository/p2,
             which some other "mvn install" may have placed
             there.
         -->
        <tycho.localArtifacts>ignore</tycho.localArtifacts>

        <!-- Issue warnings instead of aborting on error -->
        <baselineMode>warn</baselineMode>
        <jgit.dirtyWorkingTree>warning</jgit.dirtyWorkingTree>

        <!-- Skip unit tests -->
        <maven.test.skip>true</maven.test.skip>
        <skipTests>true</skipTests>

      </properties>
    </profile>
  </profiles>

  <!-- Enable the above settings -->
  <activeProfiles>
    <activeProfile>my-css-settings</activeProfile>
  </activeProfiles>
</settings>

To verify that your Maven setup uses the properties as you define them in your settings file, you can use these commands in any of the CSS source tree locations that contain a Maven pom.xml:

# Should list "my-css-settings"
(cd maven-osgi-bundles; mvn help:active-profiles)

# In the long output, locate the <properties> and
# verify they include what you entered in settings.xml
(cd maven-osgi-bundles; mvn help:effective-pom)

Finally, to build CSS from sources, run Maven like this:

# To start fresh, clean your local repository
# If you have accidentally invoked
#   mvn install
# or want to assert that you start over fresh,
# delete the Maven repository:
rm -rf $HOME/.m2/repository

# If you want to compile the maven-osgi-bundles
# and listed its local repo in your composite repo,
# do it.
# Otherwise skip this step, and use the only repo
# for this module.
(cd maven-osgi-bundles; mvn -B -P my-css-settings clean verify)

# Similarly, compile cs-studio-thirdparty unless
# you prefer to use its binaries from a remote repo.
(cd cs-studio-thirdparty; mvn -B -P my-css-settings clean verify)

# Again if you prefer local mud
(cd diirt; mvn -B -P my-css-settings clean verify)

# If you want to compile core, ..
(cd cs-studio/core; mvn -B -P my-css-settings clean verify)

# If you want to compile applications, ..
(cd cs-studio/applications; mvn -B -P my-css-settings clean verify)

# Compile desired products
(cd org.csstudio.product; mvn -B -P my-css-settings clean verify)
# or:
(cd org.csstudio.sns; mvn -B -P my-css-settings clean verify)
# or:
(cd org.csstudio.nsls2.product; mvn -B -P my-css-settings clean verify)

After the last step, the created products can be found in the following locations, depending on which products you built:

org.csstudio.product/repository/target/products
org.csstudio.sns/repository/target/products
org.csstudio.nsls2.product/repository/target/products

Most products also create a P2 update repository from which the generated product can install optional features, for example

org.csstudio.sns/repository/target/repository

After changes to the source code, you do not need to recompile everything. If you change for example code under cs-studio/applications/logbook, you can build only that sub-module of applications via

(cd cs-studio/applications/logbook; mvn -B -P my-css-settings verify)

<child location="somewhere/cs-studio-eclipse-4.3/p2repo"/>
<child location="somewhere/cs-studio-4.4/p2repo"/>

Fundamentally, using the Eclipse Integrated Development Environment (IDE) for CSS development is easy:

The IDE allows you to run the products in debug mode, setting breakpoints, examining the source code.

Once the product is acceptable for use at your site, you can export it from the IDE and install the result on the desired computers.

You need

Both the JDK and the RCP IDE need to be a specific version, ask other CSS developers for the currently supported versions.

Start the Eclipse IDE, and assert that it uses the JDK:

The Eclipse IDE includes Maven support, but not Tycho. Since the CS-Studio sources are now based on Tycho pom.xml files, the IDE needs to create the original IDE files when the sources are imported. Add Tycho support to the IDE:

The IDE may still not understand all the maven commands found in the pom.xml. This is not a problem, because several commands are only meant for maven execution outside of the IDE. To ignore such errors:

Even if you import the complete CSS source code into the Eclipse IDE, you still need certain external dependencies that are not included in the source code. The Maven Build obtains external resources from online repositories listed in the pom.xml files. While the Eclipse IDE reads the pom.xml files to some extend, it will not fetch external resources.

An Eclipse Target Platform lists a set of plugins that the IDE uses when compiling source code. By default, the target platform of the IDE includes only the IDE itself. For CSS development, you need a target platform that includes the following:

You only need to import the sources on which you want to work. For example, you can import just the product sources, then start the product within the IDE. All its dependencies, for example required cs-studio/core plugins, will be obtained from the target platform.

If you want to work on plugins from cs-studio/applications or cs-studio/core, you can later import them as needed.

To import the desired sources into the IDE:

In the Eclipse Package Explorer, select the view drop-down menu Top Level Elements and pick Working Sets.

The IDE should now compile the imported sources. There should be no compilation errors.

Open one of the *.product files, for example

org.csstudio.product/repository/cs-studio.product

or

org.studio.sns/repository/basic-epics.product

You can use the Eclipse Search, File menu to locate all available product files as shown in Figure 4.1, “Locating all Product Files”

Figure 4.1. Locating all Product Files

When you open an Eclipse product file, it will be displayed as per Figure 4.2, “Eclipse Product Editor”. In the Overview tab, first press Synchronize, then press Launch an Eclipse Application to run the product from within the IDE. The product should start up.

Figure 4.2. Eclipse Product Editor

When running a product from within the IDE for the first time, the main goal is that the product should start. It may then stop because of missing command line arguments, but there should be no errors regarding missing plugins or compilation problems.

The product may not start up, instead reporting errors like “Product ....could not be found” or “Missing required bundle ...”, even though the Maven Build resulted in a product that starts without errors. This results from the Maven Build automatically including required plugins, while the IDE expects each plugin that you want to include in the product to be listed in the product file or its features.

Ideally, all products would include their dependencies such that the same product file can be used by the Maven build as well as the IDE without changes. Meanwhile, you can invoke the menu Run, Run Configurations... Locate the configuration for the product, open its “Plug-ins” tab. Press Validate Plug-Ins to identify unresolved plugins, i.e. missing dependencies. Either manually add the plugins that were reported as missing, or try the Add Required Plug-ins button to add them automatically. While the Add Required Plug-ins button is easier to use than the manual approach, it tends to overzealously add test fragments to the procuct which are not actually required.

In the case of the command-line products like the Alarm Server it might soon exit with an error message indicating that it requires command-line arguments. It is of course trivial to provide command-line arguments when invoking such tools as intended from the shell. During development and initial testing, though, it can be more convenient to execute CSS products from within the IDE. To add command-line arguments to products executed in the IDE, invoke the menu Run, Run Configurations... Locate the configuration for the product, open its “Arguments” tab. The “Program Arguments” section may already contain entries like -os ${target.os}. Add your desired command-line arguments to the end of the program arguments.

Once you get the product to run: Congratulations! Almost as easy as running the product is debugging it: Instead of running the product again from the menu Run, Run History, you select Run, Debug History to start the product in debug mode. You can browse the source code, set breakpoints by double-clicking at the start of a source line, then step through the code from the breakpoint on.

The Eclipse IDE has good support for executing JUnit tests and test-driven development in general. You can start many programming tasks by first implementing the JUnit test, using the IDE “Quick Fix” feature to create skeletons for the required classes and interfaces, which you then fill with the actual code until the test passes.

You will find several JUnit tests in the CSS plugin code because a lot of it was implemented in a test-driven fashion. These classes are often found in a test/ source folder, and the source files will be named *Test.java or *Demo.java.

To execute a JUnit test, you simply right-click on the class file in the Project Navigator and select Run As, JUnit Test. Some tests will need configuration files to specify database URLs or other site-specific parameters. Refer to comments in the source code for details.

Eclipse supports a special type of JUnit test to allow testing within the Eclipse runtime environment. This is necessary for tests that depend on the Eclipse plugin registry and preference system. Tests that require the plug-in runtime should be in source files named *PluginTest.java or *HeadlessTest.java. To execute them, select Run As, JUnit Plug-in Test from the file context menu.

When you invoke such a JUnit Plug-in Test for the first time, the IDE will create a run configuration that loads all the plugins found in your workspace. If the plugin containing your test refers to Eclipse user interface plugins, the plugin test configuration will in addition start an instance of the complete Eclipse IDE with all your workspace plugins, then execute the test. This can take a long time and be overkill for what you want to test.

A “Headless” plugin test is a test that requires the Eclipse plugin runtime environment but not the complete IDE user interface. Such test source files are typically named *HeadlessTest.java. To invoke such a test, you can once execute it via Run As, JUnit Plug-in Test to create the basic run configuration, but then you should edit it as follows:

End users of CSS cannot be expected to open the IDE, install the source code, and then run CSS from within the IDE. They need a product that executes from their “start” menu or a desktop link. The preferred way to create standalone products is via the Maven Build described earlier in this chapter, because that results in a complete set of products for all target architectures. Occasionally, however, it can be faster to create a single standalone product for testing by “exporting” the product from the IDE.

Figure 4.3. Product Export Dialog

To export a product, open the product file that was already shown in Figure 4.2, “Eclipse Product Editor” and press the Eclipse Product export wizard link in the Overview tab. This will open the Eclipse product export dialog, see Figure 4.3, “Product Export Dialog”, where you enter the following:

You should now have a directory like /usr/fred/CSS/bin/MyCSS-4.0 that contains a css executable. You can copy that directory to other computers and run it there. One method of deployment would be to offer a ZIP file of the exported product on a web page.

If you left the option to “Generate metadata repository” checked, Eclipse will generate also an update repository, see Chapter 18, Update Repository.

copy \path\to\eclipsec.exe AlarmServer.exe

The product that you export from the IDE is by default limited to the operating system on which it was exported because of the OS-dependent launcher and SWT libraries, see the section called “Java, Eclipse, RCP”. To export code for different platforms, you need the Eclipse “Delta Pack”. With the Delta Pack installed, Eclipse on OS X can for example build products for Windows, Linux, and OS X. Same for Linux and Windows.

To obtain the delta pack:

To use the delta pack:

When you now export a product from the IDE, there will be a new option “Export to multiple platforms”.

A side effect of the headless build and a P2-managed product is that it is no longer possible to simply replace plugins in a product with new versions. If you replace a plugin JAR file with a different version, even if the names exactly match, P2 will recognize the change because of checksums. It will refuse to load the modified plugin, because it was part of the original configuration. In the spirit of maintaining a well defined product with known content, this makes sense. Occasionally, however, it is a big nuisance: Fixes to small bugs are no longer possible via basic plugin updates. Instead, you have to create a new product by incrementing all version numbers, perform a complete headless build, then publish the new product and its repository on the update site.

A feature patch can be used to update only part of a product while maintaining full configuration control. Assume our current product contains a plugin org.csstudio.trends.databrowser2 with version number 3.0.1 that we intend to replace with a newer version 3.0.2. We need to determine which feature provides that plugin in the product-to-update, and we need to determine its exact version, including a possible date/time qualifier. If you do not know which feature provided the original plugin, search the files in the features subdirectory of the installed product. Assume we find that the plugin was provided as part of the feature

org.csstudio.trends.databrowser2.feature_3.0.1.20110715

Create a new Feature Patch project in the Eclipse IDE:

You can now export the feature patch, using an archive file like databrowser3.0.1.patch.zip as the target, and then use that ZIP file to install the patch into your product via the menu item Help, Install New Software... by adding the ZIP file to the available software sites.

As a result, the original org.csstudio.trends.databrowser2 plugin is replaced with the new one, while P2 remains fully aware of what version of which plugin was installed from where, so it will for example allow you to un-install the patch, or later add additional feature patches.

The Headless Build is similar to the Tycho/Maven Build, but using older tools. Originally, it was the standard way to compile CSS source code from the command line, for example as part of nightly builds. It has been replaced by the Tycho/Maven Build.