At the MarkLogic World user conference the company showed off MarkLogic 9 for the first time. This is a preview of the forthcoming release and the announcement as part of the opening keynote was designed to excite delegates.
According to Joe Pasqua, executive vice president, Products, MarkLogic Corporation: “Our global customers have the most demanding applications in the world and they’ve helped us shape what will be a major step forward in the way databases are used and the value they provide.
“Customers need to leverage their data assets and they have to do it quickly, efficiently, and adaptably because they need results today and know the questions they need to ask of their data will change tomorrow. MarkLogic 9 has fantastic new capabilities and features that will make their jobs easier and their results better.”
New features promise a lot
MarkLogic has set out some of the key features in this preview release. They are all aimed at speeding up the ability to work with multiple data sources without doing ETL, keeping data more secure and making it easier to manage MarkLogic databases.
Among the key features are:
Data Integration: Entity Services are a new feature that uses a semantic model of the key entities in the data and the relationships between them. It looks very like the contextual models we are seeing in other technologies such as IBM’s Watson and several of the graph database products. The goal is to move away from the morass of data type matching, field sizes and the challenge of working with the same data in different formats. This is where the relationships come in and it will be interesting to see what sort of speed improvement MarkLogic claims over previous releases.
The Optic API is being promoted as a new query mechanism that will speed up queries, make the complex joins that come with multiple data sets easier to create and manage while also supporting clusters. To get the best out of this will require developers and data teams to rethink and redesign their existing queries. There is no sign of a conversion engine at the moment and this will hopefully appear in the next few months. What will be interesting here will be comparisons between Optic API and the query technology used by graph database vendors. Both are claiming to be superior to standard SQL but there is a distinct need for imperical proof.
There will be tools to integrate MarkLogic with existing SQL tools so that users can use what they know and still be able to access data stored in MarkLogic. The downside of this is that there is no proof it will support all of the new Optic API features.
Security: No surprise that encryption is top of the agenda and that MarkLogic are claiming that they are making data more secure than ever before. One of the ways they are claiming to do this is through granular separation of duties. It will be interesting to see what tools they implement and how easy administrators find this to use. Previous attempts to require multiple user accounts with separate privileges have always ended up frustrating administrators and developers.
The introduction of new redaction technology to ensure that data can be protected from being displayed on screen is good news. It will certainly help to defeat screen scraping technology and MarkLogic are claiming that even if data is stolen, the redaction technology will make certain fields unusable.
Element-level security is designed to go beyond document level security and hide data from users based on their access rights and privileges. This has been a long term goal of most databases used for BI and analytics as users can often discover the data by using multiple queries to infer the missing elements. It will be interesting to see if this approach is good enough to really prevent users from accessing data they have no rights to.
Manageability: There is a new MarkLogic Ops Director which should provide administrators with a single tool to manage databases wherever they are deployed. It will be interesting to see just how well it comes with complex environments where data is spread across multiple sites and multiple clouds, especially where administrators want to move and access data in all locations.
There are two features designed to improve uptime. The first is Rolling Upgrade which will allow clusters to be updated without downtime. The second uses telemetry to do proactive alerts about the health of MarkLogic instances. One of the challenges of the latter will be how well it allows administrators to create viable SLAs across different deployment platforms from local machines to cloud.
Conclusion
MarkLogic are making some big claims about MarkLogic 9. With the product not due to ship until December 2016 there is still a lot of time for features to be deprecated although it is unlikely that much will be added. One of the goals with a preview release is to freeze the feature list to allow developers to work through what they already have and to decide what has to slip to the next version.
It will be interesting to see who MarkLogic announces as early adopters of MarkLogic 9. It already as an interesting customers base with some very large customers who have the type of complex data problem, querying across multiple data sources, that MarkLogic is addressing. There will be a lot of people keen to join the test program and one of the challenges for MarkLogic will be to manage disappointment for those who don’t make it.
This looks like a strong set of features but is it enough? Graph database vendors are already beginning to make their presence felt among customers with large and complex data sets. There will be a lot of attention on the Optic API to see if it can compete with those solutions. There is also no mention on in-memory capabilities which is also a surprise
