GeoTools

OSGeo

Friday, March 27, 2015

FOSS4GNA Code Sprint Replacing Vecmath

As Torben indicated on the geoserver blog we got together for a one day sprint after the foss4gna conference. Torben stole my picture for that post so I will have to stick to content ...

Our topic of choice ... replacing vecmath. The use of vecmath library has been a long standing "technical debt" for GeoTools. How long standing? The issue tracker number is GEOT-22.

So what is the problem with vecmath

The vecmath.jar is used by gt-referencing for coordinate system transformations. We only uses a couple classes form the library: primarily to implement GeneralMatrix for use in MathTransforms.
GeneralMatrix extending GMatrix
There is one small problem with this idea - vecmath is not open source! Technically vecmath is distributed as part of Java 3D (an extensions to the Java Virtual Machine). As an extension to Java it was distributed under the same licenses (Sun Binary License and a Research License) as Java.

With the GeoGig project going through LocationTech incubation we have a couple ways to use jars:
  • prerequisite: open source jars required to run
  • works with: optional jars that extend the functionality if present. These may be proprietary like an oracle database driver.
Although the vecmath license is fine for distribution it does not meet the strictly open source policies required for the GeoGig project. In this case we want GeoTools to include matrix math and needed to shop around for a replacement.

The use of vecmath (as a non open source dependency) also causes trouble for Rich Fecher's proposal to publish GeoTools on Maven Central. 

Enter EJML

With a technical dept page capturing research, some email discussion and a great lunchtime conversation at foss4gna with Eric Engle (the overlap with EclipseCon was good for something) we settled on the recommended Efficient Java Matrix Library (EJML).

GeneralMatrix delegating to DenceMatrix64F
The strategy here is delegate to the DenseMatrix64F implementation provided by EJML, and implement the methods we expect from XMatrix in terms of this new implementation.

The EJML library has the similar arrangement with SimpleMatrix wrapping a DenseMatrix64F an API friendly to casual developers. We were able to use SimpleMatrix as a guide, saving a lot of time.

How to help

While the code sprint was a success in proving the approach, there is a bit of work to go:
  • Tyler (from GeoGig) is working on removing the dependency on vecmath (there are a few other Exceptions and data structures in our API that need to be removed).
  • Jim (from GeoMesa) wants to write up more test cases to check for regressions between vecmath and EJML
  • Although Rich Fecher (from GeoWave) was unavailable to take part in the code sprint - his inspiration to work on this now means we will be hitting him up for a review when the work is complete. Thanks Rich!
  • And there is always the question of performance ... will it be faster!
To help take part, or review our work see this branch in github:
Thanks to Boundless victoria staff, Jim and Andrea for really getting behind this work. This kind of up keep keeps the community ticking along and helps the library be used in more places.

I would also like to thank the new crop of projects using GeoTools for taking part and contributing upstream. It is important to keep our community diverse and your participation is both welcomed and appreciated.

Thursday, March 26, 2015

CodeHaus Migration Schedule

As per earlier blog post CodeHaus is shutting down and the GeoTools project is taking steps to migrate our issue tracker and wiki to a new home.

First up I need to thank the Open Source Geospatial Foundation for responding quickly in a productive fashion. The board and Alex Mandel were in position to respond quickly and hire a contractor to work with the system admin committee to capture this content while it is still available.

I should also thank Boundless for providing me time coordinate CodeHaus migration and Andrea for arranging cloud hosting.

Confluence Migration

Is scheduled for ... now! I have taken a copy of the CodeHaus wiki and will be migrating proposals and project history. A html dump of the wiki is published at old.geotools.org so we have a record.

The new wiki is available here: https://github.com/geotools/geotools/wiki
GitHub Wiki

Jira Migration

Jira migration is scheduled for 00:00 UTC Saturday March 28th.

On Saturday all issues will be migrated to their new home (and CodeHaus issue creation will be disabled). If you wish to lend a hand testing please drop by the #osgeo IRC channel on Saturday. Harrison Grundy will be coordinating the proceedings.

We have set up a new JIRA available at osgeo-org.atlassian.net for the migration. If you need access to the new issue tracker please contact Andrea or Jody.

OSGeo Jira
As shown above a few friendly CodeHaus refugees have also been sheltered for the storm (uDig and GeoAPI).

Monday, March 23, 2015

GeoTools 13.0 Released


GeoTools 13.0 Released

The GeoTools community is pleased to announce the availability of GeoTools 13.0the first stable release of the GeoTools 13 series.

Download GeoTools 13:
This release is also available from our maven repository, and is made in conjunction with GeoWebCache 1.7.0 and GeoServer 2.7.0.

New Features and Improvements

The following new features and improvements are available in this release:

Data Stores
  • GeoPackage support is now compatible with QGIS and OGR.
  • A new gt-solr data store is available for working with with Apache Solr. Thanks to Andrea Aime and Justin Deoliveira for this work.
  • CSVDataStore has been improved with write access and the ability to work with different kinds of CSV files (including lat/lon and WKT). Thanks to Travis Brundage for this work.
  • AbstractDataStore is now deprecated. Please make plans to migrate to ContentDataStore. There is an extensive ContentDataStore tutorial to help with your migration.
  • PropertyDataStore has now been ported to use ContentDataStore, and a wide range of issues (readers, events, transactions) have been resolved for all DataStore implementations. Thanks to Torben Barsballe for this extensive QA work.
CSS Module

  • The gt-css module is a brand-new implementation of CSS-style support, now written in Java. Thanks to Andrea Aime for bringing this exciting approach to styling within easy reach of Java developers. Here is a quick code example:
Stylesheet ss = CssParser.parse(css);
CssTranslator translator = new CssTranslator();
Style style - translator.translate(ss);

Rendering Engine
  • Color-blending has been introduced as both a FeatureTypeStyle and Symbolizer vendorExtension, allowing for a range of special effects. Thanks to Cleveland Metro Parks for this improvement.
  • FeatureTypeStyle vendorExtension named "firstMatch". This vendorExtension will grab the first rule you specify. This can in many situations help you avoid more complicated logic for style rules.
    <sld:VendorOption name="ruleEvaluation">first</sld:VendorOption>

  • Anchor points are now supported. This is done in SLD with the AnchorPoint tag:
    <LabelPlacement>
      <PointPlacement>
        <AnchorPoint>
          <AnchorPointX>
            <PropertyName>X</PropertyName>
          </AnchorPointX>
          <AnchorPointY>
            <PropertyName>Y</PropertyName>
          </AnchorPointY>
        </AnchorPoint>
      </PointPlacement>
    </LabelPlacement>
WFS Client

The gt-wfs client has improved:
  • Better compatibility with MapServer.
  • Extensive work has been done to support WFS 2.0 transactions. Thanks to Niels Charlier for this work.
  • The gt-wfs client now supports WFS 2.0 stored queries. Thanks to Sampo Savolainen for this work.
And more
Documentation continues to improve, now with a complete function listFor a full list of improvements and changes in GeoTools 13, please see the full release notes.

Thanks to Jody Garnett and Travis Brundage (from Boundlessfor making this release happen.

Thursday, March 5, 2015

CodeHaus Migration

It with heavy hearts (and a small bit of panic) that we bid adieu to CodeHaus. After many years of service CodeHaus is shutting down. A bit of last moment sponsorship from sonarqube is holding the doors open giving us a chance to migrate in an orderly fashion.

This blog post is a public notice that the GeoTools project will require assistance during this migration due to the short time frames involved.

GeoTools will lock down changes to the issue tracker after the scheduled March 18th release of GeoTools 13.0.

Migration Plan

In this weeks community meeting we have set on a migration strategy.
  • Migrate GeoTools issue tracker to a new instance Jira (sorting out host options see below). This is primarily so we can manage large attachments (such as test data) that make our issue tracker valuable. As a fallback plan we will consider migration to github issues.
  • Migrate GeoTools wiki to github wiki with a focus on proposals and project history.
Similar arrangements are being made for our friends with the GeoServer (OSGeo), uDig (LocationTech) and GeoAPI (OGC) projects. If you are in position to assist these projects please contact the appropriate development list as they could use a hand.

Due to a technical limitation (the use of a single sign on system) we cannot just take a backup and restore on a new system. This leaves us with two options: Use REST API and a script to migrate to a new system, or assemble a work party to transfer issues via CSV export (and grab the attachments one at a time).

Thus far we have contacted:
  • The OSGeo board has offered their support in todays meeting for both setting up hosting and requesting a proposal for data transfer via REST API.
  • The OSGeo system administration committee are looking into options for hosting.
    Thanks to both Harrison Grundy and Alex Mandel for helping refine migration requirements on IRC.
  • Both GeoSolutions and Boundless have offered assistance
  • We have reached out to Atlassian about cloud hosting
Thank you to the above groups for their prompt assistance.

About CodeHaus

CodeHaus has been an amazing resource for our open source community, offering friendly hands on support in the face of the growing pains experienced with SourceForge. We should also thank Atlassian for making their software (Jira and Confluence) available to open source projects on CodeHaus.

Thursday, February 19, 2015

GeoTools 13-RC1 Released

The GeoTools community is pleased to announce the availability of GeoTools 13-RC1, the first release candidate of the GeoTools 13 series:

This release is also available from our maven repository, and is made in conjunction with GeoWebCache 1.7-RC1 and GeoServer 2.7-RC1.

We would like your feedback on these new features and improvements:
  • GeoPackage support is now compatible with QGIS and OGR.
  • The rendering engine has been overhauled with two new features:
    • Color-blending has been introduced as both a FeatureTypeStyle and Symbolizer vendorExtension allowing for a range of special effects. Thanks to Cleveland Metro Parks for this improvement.
    • FeatureTypeStyle vendorExtension "firstMatch" will stop at the first rule.
    • Anchor points are now supported.
  • The gt-css module is a brand-new implementation of css style support written in Java. Thanks to Andrea for brining this exciting approach to styling within easy reach of Java developers.
  • A new gt-solr data store is available for working with with Apache Solr 
  • AbstractDataStore is now deprecated - please make plans to migrate to ContentDataStore.
    • There is an extensive ContentDataStore Tutorial to help with your migration. Thanks to Travis for updating the initial tutorial used at FOSS4G.
    • PropertyDataStore has now been ported to use ContentDataStore, and a wide range of issues have been resolved during extensive QA testing (readers, events, transactions). We would like to thank Torben for this extensive work.
  • CSVDataStore has been improved with write access and the ability to work with different kinds of CSV files (including lat/lon and WKT). Thanks to Travis for this work.
  • The gt-wfs client has improved:
    • Better compatibility with MapServer.
    • Extensive work has been done to support WFS 2.0 transacations. Thanks to Niels for this work.
    • The gt-wfs client now supports WFS 2.0 stored queries. Thanks Sampo for this work.
  • Documentation continues to improve, now with a complete function list.
  • For a full list of improvements changes 13-beta, please see the Jira release notes

About GeoTools 13

Tuesday, February 17, 2015

GeoTools 11.5 Released

The GeoTools team is happy to announce the release of GeoTools 11.5 available for download.
This release is a maintenance release for the GeoTools 11 series. Check out the change log for all of the details.

This is the last release of the GeoTools 11 series. Migrate to GeoTools 12 for the latest stable release.
 
Thanks for using GeoTools!

Thursday, February 5, 2015

GeoTools 13-beta Released

The GeoTools community is happy to announce the latest  GeoTools 13-beta download:

This release is also available from our maven repository. This release is made in conjunction with GeoServer 2.7-beta.

This is a beta release of GeoTools and we would like your feedback on the new features outlined below.
  • GeoPackage support is now compatible with QGIS and OGR.
  • The rendering engine has been overhauled with two new features:
    • Color-blending has been introduced as both a FeatureTypeStyle and Symbolizer vendorExtension allowing for a range of special effects. Thanks to Cleveland Metro Parks for this improvement.
    • FeatureTypeStyle vendorExtension "firstMatch" will stop at the first rule
    • Anchor points are now supported.
  • The gt-css module is a brand-new implementation of css style support written in Java. Thanks to Andrea for brining this exciting approach to styling within easy reach of java developers.
  • A new gt-solr data store is available for working with with Apache Solr 
  • AbstractDataStore is now deprecated - please make plans to migrate to ContentDataStore.
    • There is an extensive ContentDataStore Tutorial to help with your migration. Thanks to Travis for updating the initial tutorial used at FOSS4G.
    • PropertyDataStore has now been ported to use ContentDataStore, and a wide range of issues have been resolved during extensive QA testing (readers, events, transactions). We would like to thank Torben for this extensive work.
  • CSVDataStore has been improved with write access and the ability to work with different kinds of CSV files (lat/lon, WKT, etc...). Thanks to Travis for this work.
  • The gt-wfs client has improved:
    • Improved compatibility with MapServer
    • Extensive work has been done to support WFS 2.0 transacations. Thanks to Niels for this work.
    • Can now work with WFS 2.0 stored queries. Thanks Sampo for this work.
  • Documentation continues to improve, now with a complete function list.
  • For the extensive list of improvements check the release notes
Thanks to Jody and Travis (from Boundless Victoria) for this release.

About GeoTools 13