FeaturesPluginsDocs & SupportCommunityPartners

NetBeans: AutoUpdate
NBM package technology

Version: 0.1 DRAFT
Author: Robert Novak, Ales Kemr, Sun Microsystems

Document History:
[29-Jan-2002] : version 0.1 : Initial revision
[10-Jan-2003] : version 0.2 : Documentation about L10N NBMs

Abstract

There is no complex documentation available now, that would describe using of the NBMs. This document was written as a source of information for further discussion. It will be regulary updated according the results of the discussion. The aim of the process is to documentate all activities connected with using of the NBMs, define roles of the people involved in each part of the process and prevent problems caused by lack of information.
If you have comments, find a mistake or you think that some information are incorrect, please, contact the author.
Contents:
1. What is the NBM ?
1.1. Description
1.2. Structure
1.3. Info.xml
1.4. Specification version
1.4.1 How to increase specification version

2. How to build an NBM package
2.1. Building steps
2.2. Example of building script

3. How to deliver NBM packages
3.1. NetBeans Update Center
3.1.1 Architecture
3.1.2 Making modules available for download

4. How to create your own Update Center
4.1 Summary
4.2 Simple example of Update Center Describing module
4.3 Advanced example of Update Center Describing module

5. How to create own Update Centre Server (static)
6. L10N NBM package
6.1 Summary
6.2 Examples

1. What is the NBM ?


1.1. Definition

NBM (NetBeans Module) is an package format used for non-installer distribution of NetBeans or FFJ modules usually with the autoupdate. Extra modules, module updates and patches are distributed as NBM patches, NetBeans or FFJ Autoupdate module allows users to connect to the Autoupdate server, select which modules should be installed and install them. It also allows to install NBMs from other sources like downloaded from web site, etc.


1.2. Structure

Basicaly, NBM is a jar package - directory structure packed with jar packager and renamed to .nbm. It must contain module jar file plus other module related files and info.xml file with information required by the Autoupdate module.


Typical structure of the NBM package:

module_name.nbm
    |   
    +- Info
    |   |
    |   +- info.xml
    |
    +- netbeans
          |
          +- modules
                |
                +-module_name.jar

The content of the directory "netbeans" depends on type of the package. It usually contains directory "modules" with module jar. It could also contains directories "lib" with libraries or patch jar, "system" with system jars, "doc" with documentation etc.

The name of the NBM package is the same as the name of the module jar - it is the module name. Notice - it isn't necessary the same as the IDE module name.

Info.xml file is described in the next section.


1.3. Info.xml

Info.xml file contains all information required by Autoupdate module to enable users to select and install modules to IDE. Information are in XML format with following DTD. It is generated from module manifest (it MUST contain all items from manifest file) and additional information - license, home page URL, distribution URL. See the Section 2 about NBM building for details.

List of information usually stored in the info.xml file:

  • Module specific information
    • codenamebase - module code name
    • homepage - home page URL
    • distribution - download (distribution) URL
    • license - license name
    • downloadsize - NBM size
    • needsrestart - does the module need IDE restart to be installed? (default = "true")

  • Manifest specific information (see Modules API for details)
    • OpenIDE-Module - Module code name
    • OpenIDE-Module-Name - Module real name
    • OpenIDE-Module-Layer
    • OpenIDE-Module-Install
    • OpenIDE-Module-Short-Description - Short module description
    • OpenIDE-Module-Long-Description - Long module description - text displayed when module is selected in the IDE, in the Update Center.
    • OpenIDE-Module-Specification-Version - Specification Version
    • OpenIDE-Module-Implementation-Version - Implementation Version - It usually is build number or build date.
    • OpenIDE-Module-Module-Dependencies (optional) - List of other FFJ or NetBeans modules required by the module
    • OpenIDE-Module-Package-Dependencies (optional) - List external jar packages required by the module
    • OpenIDE-Module-Java-Dependencies (optional) - List of JDK/JRE jars required by the module
    • OpenIDE-Module-IDE-Dependencies (optional) - List of IDE jars required by the module

  • License
    • name - License name
    • License text

  • Description (optional - can be part of the manifest information - item OpenIDE-Module-Long-Description)
    • Description text

  • Module notification (optional)
    • Notification URL
    • Notification text


1.4. Specification version

Specification version is a module version number that indicates there was a change implemented in the module and its size.
Specification number policy:

  • It can have at least 2 and up to 5 numbers separated with dot in format x.x.x.x.x - for example: 2.0, 1.2, 2.0.1, 3.1.2.1, 1.7.1.0.1 etc.
  • Number on the first and the second place has to be changed when there is a big change implemented in the module since the last release. For example: 1.2 -> 1.3, 1.2.1 -> 1.3, 1.2 -> 2.0
  • Number on the third place has to be added or increased if already presented, if there was a minor change implemented since the last release. For example: 2.0 -> 2.0.1, 1.2.1 -> 1.2.2
  • Number on the fourth or fifth place must be added when module update is being updated after release or increased if there was already released update for this module. For example: 2.0 -> 2.0.0.1, 1.2.1 -> 1.2.1.1, 1.7.2.1 -> 1.7.2.2

Specification version is a key information required by the Auto Update module. It is used to determine whether the module should be displayed in the IDE as available for download. Specification version of new module is compared to specification version of the installed one and module is displayed only when the new module specification version is higher then the installed module specification version.

See also Modules API for more details about versioning.


1.4.1 How to increase specification version

Module developer (module owner) is responsible for correct specification version. It must be increased as soon as any change in module appear. It's up to developer to decide how serious is the implemented change and to increase apropriate number of the module specification version.
Special case is module update. It is prepared by Sustaining, so they are responsible for the correct module specification version.

Typical situations:

  1. There was main release (FCS release) of product. Modules are changed, because new versions are being developed (new features are added etc.) ==> module specification number MUST be increased as soon as any change appears. As it was already mentioned, the way how to increase specification version depends on the type of change in the module. In this case it is usually serious change (but doesn't have to be), which requires to increase number on the first or the second place of the specification version to satisfy autoupdate module funcionality - it must be possible to distribute modules via Autoupdate.

  2. There was an preview release (EA release) of product and the product is closed for any changes except of bugfixes ==> specification number of the module MUST be increased as soon as any bugfix is implemented. It is usually minor change, so it requires to increase number on the third place of the specification version to satisfy autoupdate module funcionality, because modules are distributed via Autoupdate.

  3. Updates for several modules are being released during the time between 2 main releases ==> first three numbers of the specification version remain the same as in original module and number on the fourth place is increased or added if not preseted to satisfy autoupdate module funcionality, because modules are distributed via Autoupdate.
Practical implementation:

Good way, how to keep the correct specification version to satisfy AU module funcianality and make possible to publish modules on Update Center, is to increase it with each branching. It means, to increase last number of specification version on original branch and add .1 at the end of specification version on new branch, right after a new branch is created. For example:
Specification version of a module in main trunk before branching was 1.6. Now, new branch was created, so specification version of the modulu will be increased to 1.7 in trunk and changed to 1.6.1 on the new branch.

Notice - If module wasn't changed between 2 releases, its specification version doesn't have to be increased. It's up to module developer to decide whether to increase it or not.


2. How to build an NBM package


2.1. Building steps:


  1. Check out build and modules sources from CVS:

    cvs co all
    cvs co your_module     (only if the module is not part of the standard build)

  2. Download and unzip extra binaries:

    Extra binaries are available at web page: http://www.netbeans.org/downloads.html in section Building from sources. Choose the rigth version of extra binaries.

  3. Enter the nbbuild module directory:

    cd ./nbbuild

  4. Build module jar:

    ant -Dbinroot=extra_binaries_path -Dmodulename="your_module"

    extra_binaries_path - path to extra binaries - required only when extra binaries are not at the standard path (NetBeans 3.4 and earlier only, not required afterwards)
    your_module - module name

  5. Enter your module directory:

    cd ./your_module

  6. Build module nbm:

    ant -Dbinroot=extra_binaries_path -Dbuildnumber=implementation_version -Dkeystore=key_location -Dstorepass=key_password nbm

    extra_binaries_path - path to extra binaries - required only when extra binaries are not at the standard path (NB 3.4 or earlier only)
    implementation_version - an implementation version of the module, usually it is date. For example: 20011025
    key_location - path, where is certificate stored (NetBeans or FFJ)
    key_password - a password for the certificate

  7. Set read access right to all users:

    chmod 644 your_module.nbm


2.2. Example of building script


For inspiration you can look e.g. at build script of Autoupdate module.


3. How to deliver NBM packages


There are several possible ways how to deliver new modules, module updates and patches to users, but in each case Autoupdate Module is used to select and install them.

  1. Make NBMs available for download on the Autoupdate server (on Update Center)
    - to install such modules, user has to enter the Update Center, login to the AU server and select which modules should be installed.

  2. Make NBMs available for download on web site or FTP
    - to install such modules, user has to download NBMs to the local directory, enter the Update Center, section "Install Manually Downloaded Modules", add directory with NBMs into list of NBM sources and select which modules should be installed.

  3. Deliver NBMs on CD or other media
    - to install such modules, user has to make media available for reading, enter the Update Center, section "Install Manually Downloaded Modules", add directory with NBMs into list of NBM sources and select which modules should be installed.


3.1. NetBeans Update Center


3.1.1. Architecture

NetBeans Autoupdate server
AU server is located at the www.netbeans.org and its infrastructure is maintained by CollabNet. It consists of NBMs of all modules available for download as well as their description.

Module NBMs are stored at:
http://www.netbeans.org/download/nbms/{netbeans version}/*.nbm

Possible operations with NBMs are:

  • Download - with web browser from link http://www.netbeans.org/download/nbms/{netbeans_version}/{module.nbm}

  • Upload - scp {module.nbm} upload@netbeans.org:nbms/{netbeans_version}/

  • Listing of the server content - ssh upload@netbeans.org ls -l nbms/{netbeans_version}/

  • Remove - ssh upload@netbeans.org rm nbms/{netbeans_version}/{module.nbm}
Notice - All operations require access to account upload@netbeans.org. Only autorized people - Release Engineers and AU Engineers are provided with it.


Module descriptions are stored in XML files called Module Catalogs. You can look at Module Catalog DTD. Catalogs are located in the CVS in WWW module, in the directory www/www/updates/. Any changes in the Autoupdate setup are done by checking out this directory, implementing changes to Module Catalogs and commiting them into the CVS.

The directory www/www/updates/ is also accessible through link:

http://www.netbeans.org/updates/

URL of the Modules Catalog is in the IDE called Update Center. Its format is:

http://www.netbeans.org/updates/[beta/]{IDE_version}_{AU_version}_.xml

http://www.netbeans.org/updates/[beta/]{IDE_version}__.xml, is hardcoded in the Autoudpate Module properties bundle. AU version is added by the IDE according version of the Autoudpate client. (It isn't Autoupdate specification version.)

There are 3 Update Centers for NetBeans 3.3 and later versions - Main, Beta and Alpha. For NetBeans 3.2, there is only Main Update Center. Main Update Center contains stable versions and its base URL is http://www.netbeans.org/updates/. Beta (Alpha) Update Center contains beta (alpha) versions and its base URL is http://www.netbeans.org/updates/beta (alpha).
Modules in the Update Center can be grouped into Module Groups and multiple level groups are allowed. Actual policy how to group modules:
  • Modules, that are not part of a standard build, but are maintained, are right on the first level.
  • Modules, that are minimally or not maintained, are located in module group "Unmaintained modules"
  • If a module consists of more NBMs, those are grouped into module group with the name of the module (at any level)
See the example Modules Catalog: http://www.netbeans.org/updates/33_1.5_.xml


NetBeans Autoupdate Client
It is the NetBeans Autoupdate module, that is part of the IDE. It connects to NetBeans Autoudpate server and reads selected Update Centers related to the running version of NetBeans. User selects which Update Center to use - either Main or Beta or both. User can also change the Update Center URL by editing the Update Center property "server URL". It is possible to define user Update Center located on other place then netbeans.org - see the Section 4 for details.

After Modules Catalogs are read, the tree structure of available modules is displayed and users can select modules to install. Only not yet installed modules and newer version of modules are displayed.


3.1.2. Making modules available for download

Requirements:

Before you start to publish modules on the NetBeans Autoupdate server, make sure that you have correctly built and signed NBMs of all modules and you know:

  • NB version the modules should be available for
  • Autoupdate server the modules should be available on - Main, Beta (modules stability)
  • module group the modules belog to

Manual publishing, publishing steps:

  1. Extract info.xml file from each NBM
  2. Check out module www from CVS: cvs co www
  3. Extract info.xml file from each NBM (there is a script that perform) this automaticaly
  4. Add data from info.xml files to catalog XML into appropriate module groups
  5. Upload NBMs on the Autoupdate server
  6. Commit changed modules XML catalogs back to the CVS
  7. Perform test download and make sure that displayed information are correct
  8. Inform the module owner, that module is available for download


4. How to create your own Update Center


4.1. Summary

It is possible for module owners to create own Update Center and make it available in the IDE. At first you should of course create some server with static (like NetBeans) or dynamic (e.g. servlet) modules catalog.
Only think you need then is to create simple module, depending on autoupdate, which will define your Update Center - Update Center Describing (UCD) module. You can find two examples of UCD modules in following sections. So, the process is:

  1. Create UCD module.
  2. Create it's NBM file and deliver it - e.g. make it available for download on web site
After user once installs UCD module, new Update Center will be available in the IDE. All updates of your modules can be then maintained via your Update Center.


4.2. Simple example of Update Center Describing module

This simple example of module defines Update Center with static modules catalog - static xml file on certain URL. This modules contains no java sources, only xml layers and bundle file. There is also module build script. You should only change URL in mf-layer.xml


4.3. Advanced example of Update Center Describing module

This advanced example of module defines Update Center which uses authentification - its needed to fill username/password in dialog. This dialog will be displayed in Update Center wizard. This modules contains MyAutoupdatype.java extending autoupdate/XMLAutoupdateType.java , MyLoginPanel.java, xml layers and bundle file. Build script is also attached.


5. How to create own Update Centre Server (static)

NetBeans cvs module nbbuild contains Ant task MakeUpdateDesc for creating static xml files with Update Centre Catalog. See example build.xml file and catalog file it has created.


6. L10N NBM package

This new feature is available in development builds (it was not part of NetBeans 3.4).


6.1. Summary

It is possible to distribute standalone localized versions of modules. NBM package for localization of module (L10N NBM) is determined by:

  • module codename
  • module specification version
  • language (and/or branding) code
L10N NBM of module M (language xx) will be visible on Update Center, if module M is installed (or available on Update Center) and has no xx-localization or has xx-localization with lower specification version. L10N NBM of module M actually includes hidden dependency on module M.

There is new version of Info.xml DTD supporting this feature. In case of standalone L10N NMB manifest tag is missing and there is l10n tag instead:
  • L10N specific information
    • langcode - Code name of l10n language
    • brandingcode - Branding code
    • module_spec_version - Specification version of module
    • module_major_version - Major version of module
    • OpenIDE-Module-Name - Module real name, text will be displayed in Update Center Wizard.
    • OpenIDE-Module-Long-Description - Long description of target module - text will be displayed in Update Center Wizard.


6.2. Examples

Typical structure of the L10N NBM package (localization xx):

module_xx.nbm
    |   
    +- Info
    |   |
    |   +- info.xml
    |
    +- netbeans
          |
          +- modules
                |
                +- locale
                      |
                      +-module_xx.jar
You can also look at following info.xml example of L10N NBM package for japanese localization of mymodule.

Companion
Projects: 		MySQL Database Server   Open JDK: an Open SourceJDK   GlassFish Community: an Open Source Application Server    Mobile & Embedded Community    Open Solaris   java.net - The Source for Java Technology Collaboration   Virtual Box - full virtualizer  Open ESB - The Open Enterprise Service Bus Powered by