Archive for the ‘Software Development’ Category

MapGuide, ArcGIS Server, and Manifold – First thoughts

April 16th, 2009 7 comments

In this article I'll give a brief overview of three Internet Map Servers (IMS): ArcGIS Server 9.3, MapGuide Open Source, and Manifold. I have scoured the web for information and also talked to consultants for each tool. Please keep in mind that my viewpoint is that of a person who has only acted as an end-user of each product, not a developer.

First, why am I looking for an IMS? Well I'm glad you asked! We are nearing completion of our first "for sale" software product, Landboss. Landboss is a SaaS app that provides lease acquistion and administration functionality to the oil and gas industry. Most of my functionality gathering missions took place in our homestate of Oklahoma. Our initial plan was to support only states that use the Public Land Survey System, but after talking to some folks in TX we realized we could support them (and other non-PLSS states such as the New England area) without too much extra work.

There was a catch, as always. Because tracts of land in non-PLSS states come in all shapes and sizes, mapping is extremely important. The main reason is because users need to visualize their prospect and tracts within it. In a PLSS state you can fake this because most tracts are rectangular in nature. Thus, our potential TX clients want/need mapping in the system as soon as possible.

ArcGIS Server
ESRI is the standard in GIS, there is no arguing that point at all. The ArcGIS Server product seems snappy in all demos that I have used and has a nice look to the maps. The big advantage to ArcGIS Server is that there is a large community and a lot of support for the product and it has been tested heavily in production.

The two downsides of ArcGIS are price and multi-threading performance. ArcGIS Server is expensive for a server license if you want to open it up to the world, more than I want to pay. That problem is compounded by the fact that the ArcGIS engine product is not multi-threaded. The server itself is multi-threaded to handle requests, but once a request gets to the engine, it will not take advantage of a multi-core processor. ArcGIS server has announced native 64-bit support, but I'm not sure if it's available yet or not.

ArcGIS Server will run on Windows, Linux, or Sun Solaris.

MapGuide OS
MapGuide OS is backed by AutoDesk. They open-sourced MapGuide, I assume, to combat ESRI's stranglehold on the marketplace. AutoDesk still sells a licensed enterprise version that is built around MapGuide OS.

MapGuide's interface is nice and response time was quick. The quality of the rendering was good as well, although I don't think there is a significant difference between ArcGIS and MapGuide. The huge benefit with MapGuide is that it is open source and free to use. For what I want to do, the work is fairly straightforward and I'm looking at spending less on a consultant than I would spend on one license of ArcGIS Server!

The main reason the consultant's time would be less is because MapGuide comes with quite a bit of functionality out of the box, like tiling, AJAX, and more. It also has support for all major spatial databases.

MapGuide OS will run on Linux or Windows.

Manifold is a very interesting product. It claims huge performance benefits on it's website and shamelessly bashes ESRI. I have no reason to believe that those performance benefits are false and from reading and discussing with a consultant I have reason to believe they are true. I was very intrigued by their support for NVIDIA CUDA and in future versions they plan to move more function calls to CUDA (currently it's a limited number of functions that can run in CUDA).

The maps themselves are good quality and the rendering speed was good. It would be hard to compare the three products on performance without doing a formal benchmark of each under various loads.

As a desktop GIS product I think Manifold is extremely competitive because of it's pricing and capabilities when compared to ESRI. The IMS falls down short though because the out of the box implementation is horrible. To get to a nice feature set that MapGuide has out of the box would take you hundreds of hours of developer time. The performance benefits are simply not good enough to overcome the extra cost of development time. Manifold does not seem to have any interest in building out the server product, they seem content to focus their time on the engine.

Manifold proudly flies the Microsoft flag and even suggests that supporting only one platform is a benefit because it makes their job simpler.

And the winner is...
If you haven't already guessed it, MapGuide OS is the direction we have chosen. For the quality and features of the software you simply can't beat it when it is free and open source. You can interface with the product using multiple languages (we prefer C#) so it fits into most development environments easily. AutoDesk still supports MapGuide heavily so it is far from a dead product and there is a large support community around the product in case you need help developing your map application.

Once we put MapGuide into production I will post a follow-up and let everyone know how it goes.

Castle ActiveRecord and Eager Fetching

January 10th, 2009 5 comments

In the beginning of 2006 we began using NHibernate at InterWorks. The learning curve to NHibernate is steep and I quickly searched for any tools or frameworks that could mitigate the learning curve by simplifying the mapping and querying process. I found ActiveRecord at that time, but it was still a work in progress and I didn’t feel it was safe to use in production projects.

Thus began our journey to create our own framework around NHibernate. We created something very similar to ActiveRecord, but we did not hide the entire NHibernate implementation as ActiveRecord does. Instead we created a base object class (creatively named DataObjectBase) and wrote a MyGeneration template to generate our mapped objects from a database. We have this refined to the point where we can design our database and easily generate dataobjects. That template generates several convenience static methods on the dataobjects: GetById, GetByProperty, GetByCriteria, and others.

Being the kind of person that hates to do something manually twice, I’m constantly looking to automate processes. Lately the issue I’ve been thinking about is how we can more easily track and maintain database changes across version of projects. One of our guys is a former RoR evangelist (he still likes Ruby, but realizes that Rails is not a silver bullet ?) and brought up the Rails migration feature. We began to search for a migration framework to use with our NHibernate framework. During the course of that search, I found ActiveWriter for Castle ActiveRecord. A nice tool to do exactly what we need… if only we used ActiveRecord!

This prompted me to take another look at ActiveRecord. We won’t be able to move to it during this coming year, but in 2010 I’d like to move us back to a framework that has community support, be that ActiveRecord or ADO.NET Entity Framework (would have to be next release of VS.NET, current version is too limited). This prompted me to take a second look at ActiveRecord and examine if there were features missing that would keep us from using it and if so were they surmountable issues that we could implement ourselves?

One of the common arguments I have heard against ActiveRecord is that it oversimplified the mapping of objects to tables and did not allow enough flexibility to do complex mapping. That issue doesn’t bother me because we only use our NHibernate framework on new databases and have a standard database design approach that should easily work with ActiveRecord. As long as it can support joined subclasses (which it does), we are good on that front.

One issue that was tricky for us to get right with our framework was lazy loading. We tried setting lazy loading in the mapping, but quickly realized this was problematic. The data you want to lazy load is usually dependent on the context of where you are using the object and since you have no context at the point of the mapping, you simply have to make a decision based on the most likely case. Luckily, NHibernate allows you set a relationship to eager load either through HQL or the Criteria queries. You can do this by calling ICriteria.SetFetchMode(association,FetchMode.Eager) on the SubCriteria object that represents the join you want to eager load.

This would be problematic with our framework because we were trying to minimize how often the application developer had to use the Criteria object, preferring them to use our convenience methods. To resolve this we added another parameter to our convenience methods called associationsToLoad. This string would be parsed by our framework and traverse the Criteria object tree and set the specificed associations to be eagerly loaded. Here’s the code for the method that does the heavy lifting:

  2. /// Adds all the specified associations to the ICriteria. This will cause these associations
  3. /// to be loaded in one database query with the top level object.
  4. ///
  5. /// The associationsToLoad string should be in the format "association1,association2".
  6. /// The associations themselves can have multiple depth, e.g. "Company.Clients.State".
  7. /// Associations can duplicate safely, e.g. "Company.State,Company.Orders".
  8. /// All associations are set to load using a left outer join.
  9. public static void AddAssociations(ICriteria criteria, string associationToLoad)
  10. {
  11. if (String.IsNullOrEmpty(associationToLoad))
  12. return;
  14. Dictionary<string, ICriteria> subcriteriaMap = GetSubcriteria(criteria);
  15. List<string> aliases = new List<string>();
  17. string[] associationStrings = associationToLoad.Split(',');
  18. foreach (string associationString in associationStrings)
  19. {
  20. string[] associations = associationString.Trim().Split('.');
  22. ICriteria parentCriteria = null;
  23. ICriteria subcriteria = null;
  25. string association = "";
  26. string absolutePath = "";
  28. for (int i = 0; i < associations.Length; i++)
  29. {
  30. association = associations[i];
  32. if (i == 0)
  33. {
  34. parentCriteria = criteria;
  35. absolutePath = association;
  36. }
  37. else
  38. {
  39. parentCriteria = subcriteria;
  40. absolutePath = String.Format("{0}.{1}", absolutePath, association);
  41. }
  43. if (subcriteriaMap.ContainsKey(absolutePath))
  44. subcriteria = subcriteriaMap[absolutePath];
  45. else
  46. {
  47. subcriteria = parentCriteria.CreateCriteria(association, GetAlias(association, aliases), JoinType.LeftOuterJoin);
  48. subcriteriaMap.Add(absolutePath, subcriteria);
  49. }
  51. parentCriteria.SetFetchMode(association, FetchMode.Eager);
  52. }
  53. }
  54. criteria.SetResultTransformer(CriteriaUtil.DistinctRootEntity);
  55. }

This works great. It allows you to quickly optimize your page by passing in the string without having to create a Criteria query and manually set FetchMode.Eager on each association.

Unfortunately, ActiveRecord does not currently support anything like this. You have to jump down to HQL to preload associations outside of the mapping. This is just way to onerous for me. Call me lazy, but one of the major reasons to use an ORM was so you don’t have to write SQL, and to me writing HQL isn’t much better. However, I think that this would be relatively easy for us to implement on top of the ActiveRecord framework. If not, we can write it into the framework itself and submit to the community for approval.

After further review, I didn’t see any other issues that would keep me from using ActiveRecord at this time. It has an active community and appears to be a very healthy project. However, this move will be a large move for us and I will abstain my decision until I see the new version of the ADO.NET Entity Framework in VS.NET 2010.

Categories: Software Development Tags: