[Edserplo_users] Major update to Edserplo

[Steve Loch]

Maps
--------
Simon has been working on a WVS (World Vector Shoreline) version of the map as an alternative to ESRI. This was work that was initiated by Jonathan just before he left. The benefit to us is that our collaborators can use Edserplo outside these walls (e.g. BAS) without concerns over ESRI licensing. Simon has also added some GEBCO contorurs for 200 and 1000 metres and this is available for  both ESRI and WVS. Unfortunately the contours are missing from the track map but this will soon be rectified.   

Until I do some more tests you cannot see the WVS usage. The intent at the moment is to show that the existing mapping system is unaffected by the development and then we will switch to use WVS throughout with ESRI as a backup.

Incorporation of Matlab Algorithms
------------------------------------------------------
To move more quickly through data - 2D in particular - we need to adopt algorithmic screening techniques set in close association with our visualisation capability. The data scientists have been pooling ideas on this and there is quite a lot of documentation at  M:\BODC_DOC\Meetings\Internal Meetings\Edserplo development.

Rather than making  Edserplo development a bottleneck and Edserplo itself increasingly difficult to maintain I wanted to develop a technique whereby we could execute algorithms developed or expressed in Matlab without having to translate them first into Java. This has some rather obvious advantages like immediacy and decoupling Matlab and Java development. There are two approaches to this, not necessarily exclusive, that we are looking at.  One is to see Edserplo as a visualisation engine in the Matlab context ; the other is to create a Java wrapper for a Matlab engine for the Edserplo context. 

The MathWorks actually provide the latter for C and Fortran and there is a product called JMatLink, not supported by The MathWorks, which does the same for Java and which we are endeavouring to make work in BODC.   The RMI technique, whereby two programs talk to each other peer-to-peer, is proved in concept  and has some promise but will need rather more effort in its development if we go down that route. The two programs (one Edserplo, the other Matlab) don't even have to be on the same machine.


Formats and Transfer
---------------------------------
There has been concerted attempt to revise the User Guide documentation  relating to the formats encoded within Edserplo. In particular I would draw the attention of Transfer programmers to the additional accessor methods provided for P* (a NOCS format). Within Matlab you can find these by using methodsview on the class (methodsview uk.ac.bodc.formats.FormatPSTAR). 

To use the methods you have to extract the  so-called accessor (the Format access object) for the Series out of the Series. An example will suffice.

[chans,sr]  =  pshead(filnam);
platDet  =  char(sr.getAccessor.getPlatformDetails)  %Returns three Java Strings as a 2D character array

The ins and outs of the  methods are described in the User Guide which can be accessed either on the PC or from an Exceed session using 
edserplo --help (I think it's quicker and it's OK to say OK (!) when you kill the window as the client continues).

FormatPSTAR has been enhanced to deal with duplicated channel names (apparently PEXEC doesn't check for this). It renames the later channels. Read about it in the User Guide.

Additional work is still needed on TASK, ACCLAIM and Sea-Bird. Simon and I are in the process of adding a CSV format which will provide a link with spreadsheets.

Tidal data
---------------
For people who have to dig into the (POL) archives of tidal data the following may be of interest.

I have recently  discovered that TERP data  (binary format) can actually pass a year boundary.  The code and documentation has been updated. 

In researching another ancient tidal processing format supported by Edserplo,  A1D (24 values/80bytes), I discovered that some of the files with .A1D extensions were in fact BCM. A1D has 24 values per 80-byte record whereas BCM has 12. Edserplo ignores extensions.

Tidal Processing
--------------------------
With a fresh Oracle ID and in the context of POL a synonym is created for BODC.PORTL in the ID.
 
ADCP
---------
Until recently all ADCP accessed in a batch have, it seems, had the same number of bins. This is not a requirement but a limitation - which has now been removed.


Monitoring NFS
------------------------
We have made a change to circumvent the timeout that was affecting the insertion of data to one of our tables.

Other Items
------------------
The 'O' option for saving settings should be friendlier now.

Java 1.6
--------------
We're still stuck at Java 1.5 in the development environment because we have been unable to migrate our Matlab usage to 2007a. The 2007a version of the database toolbox - an absolutely vital component of our computing infrastructure - cannot be used until the MathWorks have done some work on it. This means that LIVBOP3 cannot be used for Matlab work either. That said we're not greatly inconvenienced at the moment but we don't want to fall too far behind the leading edge.

Completed Migration from IRIX!
------------------------------------------------
The Serplo pages have to be updated in the User Guide and there are 3 format entries, as remarked above, to be completed (100K+ lines and 5.5 years work, and not just migration, almost done!). Much thanks are due to the students who have contributed to this project. Their names are on the User Guide with the exception of Arif, who I think should be, as the Analysis page is his in large degree. 

Regards. 
Steve