Apache MetaModel
Apache
MetaModel
Providing a common interface for discovery, exploration of
metadata and querying of different types of data sources. With
MetaModel you get a uniform connector and query API for:
PostgreSQL
MySQL
Oracle
SQL Server
Apache Hive
Embedded DBs
CouchDB
MongoDB
HBase
Cassandra
ElasticSearch
DynamoDB
Salesforce.com
SugarCRM
CSV files
Spreadsheets
XML
files
JSON
files
MetaModel
isn't
a data mapping framework. Instead we
emphasize abstraction of metadata and ability to add data sources at
runtime, making MetaModel great for generic data processing
applications, less so for applications modeled around a particular
domain.
News
Looking for the changelog? Check out our
CHANGES.md
file.
2021-02-23 MetaModel release 5.3.3 -
Improvements to Elasticsearch and Fixed Width connector. Upgraded dependency on PostgreSQL driver.
2020-01-06 MetaModel release 5.3.2 -
Improvements to Exel connector. Upgraded dependencies on Sprint, Jackson and Elasticsearch.
2019-09-13 MetaModel release 5.3.1 -
Improvements to JDBC and HBase connectors and CSV reader.
2019-04-04 MetaModel release 5.3.0 -
New connector for .arff files. Improvements to JDBC and HBase connectors. Array/list-support in the MAP_VALUE function.
2019-01-04 MetaModel release 5.2.0 -
Added support for all the major Java versions: 8, 9, 10, 11! Upgraded dependencies on Guava, Hadoop, POI and Cassandra.
2018-06-28 MetaModel release 5.1.0 -
New and updated connectors, plus updates to improve our APIs.
[METAMODEL-1188] - Added write-support for the Apache HBase connector.
[METAMODEL-1179] - Refactored ElasticSearch REST connector to use new official REST based client from Elastic.
[METAMODEL-1177] - Made TableType.TABLE the default table type, replacing null values.
[METAMODEL-1185] - Added a new connector for Apache Kafka.
[METAMODEL-1184] - Improved support for data/column types in Neo4j connector.
2017-12-23 MetaModel release 5.0.1 -
Important bugfixes around querying MS SQL Server, MongoDB and Salesforce.com connectors. Updated to latest version Apache POI dependency for Excel connector.
2017-11-13 Membrane release 0.1 -
The
Membrane subproject
has been released! Try it easily using our
Docker Hub image
2017-11-08 MetaModel release 5.0.0 -
We're excited to announce the availability of Apache MetaModel 5.0.0! Here are the highlights of this big milestone for the project!
[METAMODEL-6] - Added update summary containing information about changes on returning UpdateableDataContext.executeUpdate(..)
[METAMODEL-7] - Applied collections, lists etc. instead of arrays in the MetaModel API.
[METAMODEL-222] - Added support for Java 8 lambdas, removed support for Java 7.
[METAMODEL-1139] - Employed Java 8 functional types (java.util.function) in favor of (now deprecated) Ref, Action, Func.
[METAMODEL-1151] - Added DataContextFactory classes for instantiating DataContexts of many types based on properties.
[METAMODEL-1165] - Added a convenient alias table "default_table" for single-table data stores.
[METAMODEL-86] - Made queries inside of a JDBC UpdateCallback pertain to the running transaction.
Plus a lot more - see the
changes file
for details.
2017-07-23 New subproject Membrane -
A new subproject has been created: Apache MetaModel Membrane. It's purpose: Data Federation as a RESTful service. Visit
the GitHub page
to contribute.
2017-05-18 Early access release 5.0-RC1 -
We're making available this release candidate since the 5.x development has reached a milestone at which the several major
API changes
are code complete.
2017-02-03 MetaModel release 4.6.0 -
New connector for Amazon DynamoDB. New NOT IN and NOT LIKE operators. Made PojoDataContext thread-safe.
2016-11-16 MetaModel release 4.5.5 -
Optimized lookup by primary key in Cassandra. ColumnNamingStrategy support for CSV and Fixed Width files. Bugfixes. Better Apache Hive support through JDBC driver quirks.
2016-08-04 MetaModel release 4.5.4 -
New dynamic factory SPI for DataContext objects. Cassandra 3.x support. PostgreSQL 'json' datatypes support. Added EBCDIC file support.
2016-07-03 Mailing list update -
We've added two new mailing lists:
issues@metamodel.apache.org (for issue/JIRA notifications), and
user@metamodel.apache.org (for users of MetaModel to ask questions).
2016-05-19 MetaModel release 4.5.3 -
Added ColumnNamingStrategies for CSV, Excel and Fixed Width files. Several performance and functional fixes for ElasticSearch and JDBC.
2016-03-22 MetaModel release 4.5.2 -
Fixed query serialization bug. Fixed DB2 query rewriting issue having an OFFSET but no LIMIT.
2016-02-12 MetaModel release 4.5.1 -
Two new separate modules for MongoDB 2.x and MongoDB 3.x to retain compatibility with old and new APIs. Bugfixes for 4.5.0.
2015-12-17 MetaModel release 4.5.0 -
New connectors for Neo4j and ElasticSearch via REST protocol. New functions: FIRST, LAST, RANDOM and MAP_VALUE.
2015-10-30 MetaModel release 4.4.1 -
MongoDB 3.x compatibility, bugfixes in query parser, SQL timestamp precision and addition of "APPROXIMATE" keyword
2015-10-12 MetaModel release 4.4.0 -
Pluggable functions. Scalar functions. Dropped Java 6 support. Improved memory footprint in the Excel module. Multiple bug fixes.
2015-08-17 MetaModel release 4.3.6 -
Added support for Apache Hive. HdfsResource and FileResource working with directories.
2015-06-23 MetaModel release 4.3.5 -
New Hadoop module with HDFS connectivity (HdfsResource) for CSV files, Excel spreadsheet and JSON files. Added Cassandra COUNTER and PostgreSQL DOUBLE data type support. Fixed bug related to refreshSchemas().
2015-05-24 MetaModel release 4.3.4 -
MongoDB LIKE operator support. ElasticSearch complex mapping fix. Automatic column type conversion in JDBC module. Bugfixes in query parser.
2015-04-16 MetaModel release 4.3.3 -
Compatibility (ElasticSearch 1.4, HBase 1.0) updates and composite primary key support for JDBC module.
2015-03-15 MetaModel release 4.3.2 -
ElasticSearchDataContext is updateable. Bugfix in DISTINCT interpretation and lower-case query keywords.
2014-12-14 MetaModel release 4.3.1 -
First TLP release,
delivering bugfixes to the 4.3 version
2014-12-09 Apache Software Foundation announces Apache
MetaModel as new Top Level Project (
read
more
).
2014-11-37 New TLP infrastructure available -
Updated
mailing lists, git repository location, website.
2014-11-24 MetaModel release 4.3.0-incubating -
Introducing
ElasticSearch and Apache Cassandra modules.
2014-11-20 Graduation resolution accepted
2014-10-13 New Committer: Alberto Rodriguez
2014-07-28 MetaModel release 4.2.0-incubating -
Introducing
JSON file module. Made column data types pluggable.
2014-06-30 New Committer: Tomasz Guzialek
2014-05-11 MetaModel release 4.1.0-RC1-incubating -
Introducing
Apache HBase and Spring Framework modules.
2014-01-27 MetaModel release 4.0.0-incubating -
Updates to
naming of file-based schemas and migration towards Apache
namespace.
2013-06-12 Project enters incubation.
Source:
Latest source code release of MetaModel:
MetaModel-5.3.3-source-release.zip
Release Notes
OpenPGP
MetaModel-5.3.2-source-release.zip
Release Notes
OpenPGP
MetaModel-5.3.1-source-release.zip
Release Notes
OpenPGP
MetaModel-5.3.0-source-release.zip
Release Notes
OpenPGP
MetaModel-5.2.0-source-release.zip
Release Notes
OpenPGP
MetaModel-5.1.0-source-release.zip
Release Notes
OpenPGP
MetaModel-5.0.1-source-release.zip
Release Notes
OpenPGP
MetaModel-4.6.1-source-release.zip
Release Notes
OpenPGP
MetaModel-4.5.5-source-release.zip
Release Notes
OpenPGP
MetaModel-4.4.1-source-release.zip
Release Notes
OpenPGP
MetaModel-4.3.6-source-release.zip
Release Notes
OpenPGP
Binary:
A convenient package with all MetaModel modules and
dependencies as JAR binaries:
MetaModel-5.3.3-packaged.zip
OpenPGP
MetaModel-5.3.2-packaged.zip
OpenPGP
MetaModel-5.3.1-packaged.zip
OpenPGP
MetaModel-5.3.0-packaged.zip
OpenPGP
MetaModel-5.2.0-packaged.zip
OpenPGP
MetaModel-5.1.0-packaged.zip
OpenPGP
MetaModel-5.0.1-packaged.zip
OpenPGP
MetaModel-4.6.1-packaged.zip
OpenPGP
MetaModel-4.5.5-packaged.zip
OpenPGP
MetaModel-4.4.1-packaged.zip
OpenPGP
MetaModel-4.3.6-packaged.zip
OpenPGP
Maven dependency
Apache MetaModel is also distributed through the central Maven
repository. Here's your typical
Source code
The source code for Apache MetaModel is available through this
Apache Git repository:
Mailing lists
To get involved with Apache MetaModel, start by joining our
mailing lists and engage in the conversations!
Developer List
Unsubscribe
Post
Archive
User List
Unsubscribe
Post
Archive
Issue (JIRA) notifications
Unsubscribe
Post
Archive
Commits
Unsubscribe
Post
Archive
Contributing
Please refer to our
CONTRIBUTE.md
file for details on contributing to Apache MetaModel.
Issue tracking
Issues for Apache MetaModel are tracked through this Apache
JIRA system:
Examples
Query with MetaModel
With MetaModel you use a type-safe SQL-like API for querying
any datastore:
DataContext dataContext = DataContextFactory.create[TypeOfDatastore](...);
DataSet dataSet = dataContext.query()
.from("libraries")
.select("name")
.where("language").eq("Java")
.and("enhances_data_access").eq(true)
.execute();
The MetaModel query API allows you to use the power of SQL,
even on data formats such as CSV files, Excel spreadsheets, NoSQL
databases and more.
Update with MetaModel
MetaModel lets you do CRUD operations on arbitrary datamodels,
also in a type-safe manner. Batch updates and transactions are
logically modelled as UpdateScript closures.
dataContext.executeUpdate(new UpdateScript() {
public void run(UpdateCallback callback) {
// CREATE a table
Table table = callback.createTable("contributors")
.withColumn("id").ofType(INTEGER)
.withColumn("name").ofType(VARCHAR).execute();
// INSERT INTO table
callback.insertInto(table)
.value("id", 1).value("name", "John Doe").execute();
callback.insertInto(table)
.value("name", "Jane D.").execute();
// UPDATE table
callback.update(table).value("name","Jane Doe")
.where("id").eq(2).execute();
// DELETE FROM table
callback.deleteFrom(table).where("id").eq(1).execute();
});
The rest of the API should reveal itself through using the
DataContext!
Javadocs
, wiki, mailing
lists and other resources help too of course, so check them out.
Wiki
Check out the
MetaModel
wiki
for more in-depth examples and documentation.
Heritage
MetaModel was initially developed and released by Human
Inference since 2011. In July 2013 MetaModel joined the Apache
Incubator and in November 2014 MetaModel graduated to become a Top
Level Project (TLP) of The Apache Foundation. It is licensed under
the Apache 2.0 license.