DataScript

An immutable in-memory database and Datalog query engine for JS engines.

DataScript is meant to run inside the browser. It is cheap to create, quick to query and ephemeral. You create a database on page load, put some data in it, track changes, do queries and forget about it when the user closes the page.

DataScript databases are immutable and based on persistent data structures. In fact, they’re more like data structures than databases (think Hashmap). Unlike querying a real SQL DB, when you query DataScript, it all comes down to a Hashmap lookup. Or series of lookups. Or array iteration. There’s no particular overhead to it. You put a little data in it, it’s fast. You put in a lot of data, well, at least it has indexes. That should do better than you filtering an array by hand anyway. The thing is really lightweight.

The intention with DataScript is to be a basic building block in client-side applications that needs to track a lot of state during their lifetime. There’s a lot of benefits:

• Central, uniform approach to manage all application state. Clients working with state become decoupled and independent: rendering, server sync, undo/redo do not interfere with each other.
• Immutability simplifies things even in a single-threaded browser environment. Keep track of app state evolution, rewind to any point in time, always render consistent state, sync in background without locking anybody.
• Datalog query engine to answer non-trivial questions about current app state.
• Structured format to track data coming in and out of DB. Datalog queries can be run against it too.

For more info, check out project page

1.7.5 - May 4, 2025

• Enable direct-linking on JVM #487 via @raspasov

1.7.4 - Jan 5, 2025

• Patch es6 iterators not having Symbol.iterator function #479 #249 via @kasbah
• Fixed not-join not working properly with empty relations #481

1.7.3 - Jul 22, 2024

• Fixed find-datom on empty DB #477 thx @RCmerci

1.7.2 - Jul 4, 2024

• Regression: transacting many ref value as a set of inline maps #476

1.7.1 - Jun 20, 2024

• Regression: :db.fn/call returning entity without :db/id #474 #475 via @DerGuteMoritz

1.7.0 - Jun 19, 2024

• Stable sorting of sequences of various types #470
• Correctly restore :max-tx from storage
• Fixed tempid/upsert resolution when multiple tempids are added first #472
• Allow upsert by implicit tuple when only tuple components are specified #473
• Allow lookup-refs inside tuples used for lookup-refs #452

1.6.5 - May 3, 2024

• Regression: fixed some OR queries broken in 1.6.4 #468 #469

1.6.4

• Implement “constant substitution” optimization for queries #462
• Fixed :max-eid for dangling entities during reader-based serialization #463
• Fixed tempid in unique refs #464
• Pass through BigInteger/BigDeciman to freeze-fn in serializable #465 #466

1.6.3

• Fix regression in 1.6.2 #460 via @galdre

1.6.2

• Query: shortcircuit clause resolution when result is guaranteed to be empty #459 via @galdre

1.6.1

• Fixed JS version bug caused by Conn rewrite

1.6.0

• [ BREAKING ] Implement Conn as a single atom with extend-clj
  • Only breaking if you relied on getting listneres from Conn meta
  • Before: @(:listeners (meta conn))
  • After: (:listeners @(:atom conn))
• prevent explosion of redundant tuples during rule solving #457, thx @RutledgePaulV

1.5.4

• Fixed: :xform is not called on ref attributes #455

1.5.3

• Added: d/reset-schema!, d/with-schema
• JVM: Return nil from d/restore-conn if there is no db
• Do not throw from d/touch when finding hanging refs

1.5.2

• JVM: Fixed d/collect-garbage
• JVM: Return nil instead of throwing from d/restore if there is no db

1.5.1

• CLJS: Fixed shadow-cljs warning

1.5.0

• JVM: Storage! d/storage, d/store, d/restore, d/addresses, d/collect-garbage, d/file-storage, d/restore-conn. See docs/storage.md for details
• d/settings and per-database :branching-factor (passed via :opts)
• New API: d/find-datom. Works like d/datoms, but only returns single datom, but is faster than (first (d/datoms ...))
• CLJS: Optimized various parts of CLJS version related to compilation and index access

1.4.2

• [BREAKING] d/entity now returns nil for numeric eids if entity doesn’t exist #447 #448 via @lynaghk

1.4.1

• Allow the use of lookup refs in get-else and get-some #446 via @jjttjj

1.4.0

• Bump persistent-sorted-set to 0.2.3
• Improve performance of getter-fn and tuple-key-fn #438 via @bsless

1.3.15

• Performance improvement in JVM version #436 via @bsless

1.3.14

• Additionaly compare db identities when =/hash entities #433 #435 via @ggeoffrey

1.3.13

• Check for nils in upsert attributes

1.3.12

• Fixed pull with reverse component references #430

1.3.11

• Fixed serializable/from-serializable for JS interop #429

1.3.10

• Allow symbols as data #425 #426 via @zoren

1.3.9

• When parsing repeated pull attrs, last definition wins

1.3.8

• Add get to built-ins for binding on map values #423 thx @logseq-cldwalker

1.3.7

• Fixed :tupleAttrs in JS API #422

1.3.6

• Query comparison fns work for all data type and cross-types #420 #421 thx @zoren

1.3.5

• Fixed pull on filtered dbs #419

1.3.4

• Fixed pull on deserialized DB

1.3.3

• Allow symbols as constants #416 #417 via @zoren

1.3.2

• Fixed multiple reverse refs bug #412

1.3.1

• Fixed reverse recursive pulls #411

1.3.0

• New pull implementation:
  • Up to 7 times faster than old impl
  • Up to 2,5 times faster than doing pull through entities
  • Up to 4 times faster than Datomic in-memory
  • Support :xform
  • Cached parse results
  • Fixed wildcard + recursion bug: [* {:child ...}] did not work before
  • A callback to track all entities touched by pull (usable for reactive dependency tracking)
• Simplified benchmark code
• Ability to override query parse cache: datascript.query/*query-cache*

1.2.10

• Fix broken wildcard pull on non-matching ref #410 thx @joodie

1.2.9

• Allow :db/valueType :db.type/tuple #407 #408 thx @joodie

1.2.8

• Data retracted incorrectly in CLJS #404

1.2.6

• Tempid resolution fails in some cases #403

1.2.5

• Special case Inf/-Inf and NaN in serializer since JSON can’t hold them #402

1.2.4

• Allow specifying tuples in inserts/upserts when their value fully matches all the attrs in DB #400
• Bump persistent-sorted-set to 0.1.4

1.2.3

• Lookup ref should not throw in pull #315 #397 thx @timothypratley

1.2.2

• Bump persistent-sorted-set to 0.1.3 in deps.edn too #396 thx @fversnel

1.2.1

• Bump persistent-sorted-set to 0.1.3, fixes (d/conn-from-datoms (d/datoms @*conn :eavt) schema) on empty database https://github.com/tonsky/persistent-sorted-set/issues/4

1.2.0

• New namespace: datascript.serialize. Prepares database for fast to/from JSON serialization

1.1.0

• [ Breaking ] Remove ISeqable from DB/FilteredDB as it was breaking keys #393

1.0.7

• Ensure transitive property for hash value comparisons #274 #356 #386 #388 thx @filipesilva

1.0.5

• Fixed empty non-queried relation not affecting results #385

1.0.4

• Implement nav/datafy for entities #325 thx @IamDrowsy

1.0.3

• Temporary disable transient indices #373

1.0.2

• Fixed db diffs with different types of the same attribute #369 #372 thx @darkleaf

1.0.1

• Fixed upsert incorectly failing tempid check #363

1.0.0

• Support composite tuples #323
• Support return maps (:keys/:syms/:strs) in query #322 #345
• Support any hashable values in cardinality-many and indexed attributes #274 #356
• Throw when init-db is used with anything but datoms #276
• Throw on using unindexed attribute in :avet index access #344
• Throw on referencing undefined rule #319
• Fix [:db/retract e a false] working as [:db/retract e a]
• Validate tempids only used as values in transaction #304
• Throw if transacted entity id is out of range #292
• Better validate rules syntax #300
• Add keyword to the built-ins #231
• Throw when pred/fn called on unbound variable #309
• Validate inputs count in :in and d/q #297

0.18.13

• Fix empty? builtin #349, thx @ash14

0.18.12

• In or branches, only check that all free vars match, not bound ones (#348)

0.18.11

• Ignore tx when diff datoms (#347, thx @darkleaf)

0.18.10

• Support: db/retract without value (#339, #340, thx @ericdscott)

0.18.9

• Report in :tempids all newly created entities, even if they don’t have :db/id assigned in tx (#246, #337, thx @filipesilva)

0.18.8

• empty preserves meta (#331, #332, thx @darkleaf)
• Filtered db can be pprinted (#330, #333, thx @darkleaf)
• Allow aliasing map keys in pull entry (#328, thx @jamesnvc)

0.18.7

• Enable :unique/identity and upserts for cardinality-many values (#321, thx @cjsauer)

0.18.6

• Extracted pprinting to separate ns (#318, thx @zajac)

0.18.5

• Fixed core.cljc NS declaration to require cljs.reader (#316, thx @Quezion)

0.18.4

• Added re-pattern and str/blank? str/includes? str/starts-with? str/ends-with? to built-ins (#306)

0.18.3

• Fixed vector misbehaving in queries in CLJS (#262)

0.18.2

• Fixed regression when transient indexes were mutated and iterated during single tx (#294).
• Bumped persistent-sorted-set to 0.1.2 (throw if transient set is mutated and iterated at the same time).
• Fixed regression when unique ref tempid was not resolved during transaction (#295).

0.18.1

• Bumped persistent-sorted-set to 0.1.1 (Java 8 bytecode)

0.18.0

• Fixed DB and Datom pprinting (#287)
• Fixed cases when upsert resolves to tempid (#285)
• Throw on tempid in :db.fn/cas (#264)
• distinct aggregate returns set not a vector (thx @jdf-id-au)
• Ability to run tests with Kaocha
• [ BREAKING ] Some internals of datascript.arrays, datascript.btset and datascript.Datom type has changed

Performance optimizations for JVM version:

• Reimplemented btset in Java with transients and better performance
• Extracted btset to [persistent-sorted-set "0.1.0"]
• Used raw ints in Datom intead of wrapped Integers, added stored in tx sign

Numbers I get on my 3.2 GHz i7-8700B (median time per test, ms):

Tests are as follows:

For Datomic an datomic:mem:// database was used.

What we see:

• 20..40% faster transactions,
• 75% faster deserialization (db-init),
• No significant change on queries,
• JVM transactions are more than twice as fast as V8,
• JVM queries are 3-4 times as fast as V8,
• DataScript transactions are 25..70% faster that Datomic in-memory. Query times vary.

0.17.1

• or, or-join, not and not-jon support in queries (#238, #50)

0.17.0

• Implement clojure.data/diff on datascript/DB (#281)
• Drop Clojure 1.7 and 1.8 support
• Fix externs.js syntax (PR #216, thx @thheller)
• Support :as in Pull API an attr-with-opts syntax (#270, PR #271, thx @Jumblemuddle)
• Support idents expansion (PR #245, thx bamarco)
• JS API correctly handles nested maps with {":db/id"} in transactions (#228, thx @serebrianyi)
• Calling transaction fns through idents directly (PR #185, thx @refset)

0.16.9

• AOT artifacts are now deployed with classifiers (0.16.9-aot1.7, 0.16.9-aot1.8, 0.16.9-aot1.9). Main DataScript artifact has no AOTed code (related: #241, #279, #280)

0.16.8

• Docstrings for https://cljdoc.org/d/datascript/datascript

0.16.7

• Removed references to perf from datascript.query-v3 (closes #272)
• Fixed compile error under java 11 (PR #273, thx Ryan Belohlavek)

0.16.6

• Add support for renamed DB functions :db/retractEntity and :db/cas (#265, PR #256, thx Kenny Williams)

0.16.5

• Provision AOT-compiled classes in datascript.jar (closes #241)
• Fixed direct-linking compatibility (rolled back #197, fixed #219, closes #259)

0.16.4

• Support string tempids (#251, PR #252, thx Kenny Williams)
• Better compatibility with JS / CLJS 1.9.946 (aget, aset on plain objects)
• Added rseek-datoms (#253, PR #254, thx Jeremy Taylor)
• Object.equals for entites (PR #255, thx Camilo Roca)

0.16.3

• Silently skip nils in transaction (was stopping processing without an error before)

0.16.2

• Fixed compatibility with CLJS 1.9.660 (#233, #236)

0.16.1

• Fixed rules ignoring false arguments passed to them (#218)

0.16.0

• Fixed infinite loop in parser on CLJS 1.9.456+ (#210)
• Added contains? to built-ins (#211)
• Fixed handling of false values in entity cache (PR #198, thx Brandon Bloom)
• Fixed issue when string values were interpreted as lookup refs (#214)
• Speed up rschema calculation (#192, thx Andre R.)
• Optimize generated JS code by declaring fn arities (#197)
• [ BREAKING ] Removed ^:exports from datascript.core to enable dead code elimination (#191). This should only affect you if you were using DataScript from JS and were importing datascript.core directly

0.15.5

• Add vector, hash-map, array-map, list and set to built-in query functions (PR #186, thx Linus Ericsson)
• When filtering already filtered DB, filters were applied in the wrong order

0.15.4

• ~30% better performance for predicate and fn calls in queries (see #180)
• JS API now supports lookup refs in index lookups and queries (see #179)
• Lookup refs now work with :db.unique/value too (issue #171)

0.15.3

• Compatibility with 1.9 records (issue #176)
• Resolve query functions and predicates on JVM (issue #177, PR #178, thx Luke VanderHart)

0.15.2

• large long entity ids were breaking init-db (issue #165)

0.15.1

• get-else throws if nil is used for default value
• JS API: Do not keywordize attribute names in schema (PR #155, issue #153, thx typeetfunc)
• init-db sets correct max-eid when processing facts about transactions (PR #164, issue #163, thx Nick Alexander)

0.15.0

• Better error messages for upsert conflicts
• Backtracking of tempids when redefining them later with upserts (issue #76)
• Upsert works in vector form too (issue #99, #109)
• Can specify transaction number in :db/add
• Can put datoms into transaction directly (issue #121: supports both addition and retration datoms, keeps tx number)
• Added all datascript.core symbols to externs so they can be called directly from JS (e.g. in combination with mori, issue #139)
• Added reset-conn, conn-from-datoms and conn-from-db (issue #45)

0.14.0

• Unify fn binding results with existing relations (PR #138, issue #141, thx Aaron Brooks)
• [ BREAKING ] get-some returns [attr value] instead of just value (#143, same as Datomic)
• [ BREAKING ] Returning nil from query function filters out tuple from result
• Throw when calling unbound predicate/function inside a query (#111)
• Added several built-ins, including count (#142) and subs (#111)
• Lookups with nil value ([<attr> nil]) resolve to nil (#140)
• Allow pull-pattern to be specified with a input variable without ? prefix (#136, #122)
• Retract functions do not fail if lookup ref is pointing nowhere (#131)
• Fixed BTSet implementation of IReduceInit (#124)

0.13.3

• Accept nil as old value in :db.fn/cas (PR #127, thx Petter Eriksson)

0.13.2

• Fixed issue #120 with binding of large collections

0.13.1

• Guard conn? will check that value is derefable instead of checking for (instance? Atom)

0.13.0

• [ BREAKING ] Main namespace to include is now datascript.core, not datascript
• [ BREAKING ] Old datascript.core (internal namespace) was renamed to datascript.db
• [ BREAKING ] datascript.shim (internal namespace) was renamed to datascript.arrays

Motivations:

• Usage of top-level namespaces is discouraged and even generates a warning in CLJS.
• Better sooner than later.
• 0.13 seems like a great number for this sort of release.

Migration procedure:

• Just change (require '[datascript :as d]) to (require '[datascript.core :as d]) and you’re good to go.

For the sake of easy migration, there’re no other changes in this release. Just renamings.

0.12.2

• Fix null pointer exception when contains? is called with an entity (PR #114, thx Kevin Lynagh)

0.12.1

• db-init respects :db/index property

0.12.0

[ BREAKING ] Introducing new :db/index schema attribute:

• Attributes are not put to AVET by default anymore

• Following attributes are put to AVET:

  • :db/index true
  • :db/unique :db.unique/identity
  • :db/unique :db.unique/value
  • :db/valueType :db.type/ref

• All attributes put to AVET should be comparable. Note: maps and list are not comparable by default, vectors are compared value-by-value

• [ BREAKING ] Min/max aggregation functions will only work on a comaparable values

Benefits:

• You can finally store any trash easily and reliably in DataScript database: maps, vectors, functions, JS objects. It doesn’t even have to be comparable. Just do not mark it as :db/index and do not make it :db.cardinality/many
• Faster transactions. There’s ⅓ less indexes to fill
• Good defaults. Most cases where you’ll probably use AVET (lookup by value)—lookup refs, external ids, references—they are all indexed by default

Caveats:

• Your code may break (see below)
• Queries can do lookup by value event without :db/index attribute, but it’ll be slower

Migration procedure:

• Check your code for datoms, seek-datoms calls with :avet index, and index-range call
• Check your queries to see if they utilize AVET index. It happens when you use this pattern: [?free-var <constant-attr> <constant-value>]
• Mark necessary attributes with :db/index true in the schema

0.11.6

• msec argument to squuid (#95, #97)
• lookup refs in JS API pull, pull_many, entity (#94, thx Matt Senior)
• fix in Pull API for reverse non-component attributes (#91, thx Matt Senior)
• Node.js and Browser repls for dev profile (#93)
• Preconditions to validate db/conn arguments (#101)
• Id allocation bug (#66)

0.11.5

• Ported BTSet to JVM
• BTSet and BTSetIter implement ChunkedSeq
• New benchmark runner

This release brings a significant performance boost for JVM version of DataScript (numbers are in comparison to JS/v8 version):

• queries with single clause, no join: ~5–6 times faster
• queries with joins: ~3–4.5 times faster
• transact: ~3 times faster
• init-db: ~3–4 times faster
• rules: ~3-4 times faster

0.11.4

Experimental support for Clojure on JVM:

• sorted-set instead of BTSet
• polymorphic get lookup in query
• no sorting optimisation for init-db

Performance numbers so far (raw data):

• queries with single clause, no join: 30–50% faster
• queries with joins: ~50% slower
• transact: ~3.5–4 times faster
• init-db: ~2–2.5 times slower
• rules: ~2–5 times faster
• db equiv: ~5 times slower
• db hash: ~10 times slower

0.11.3

• Fixed UUID constructor warning under CLJS 0.0-3291 and later

0.11.2

• Support reverse attribute refs in combination with wildcards (issue #80)
• *.cljc sources do not conflict with Clojure now. Code compiles under [lein-cljsbuild "1.0.6"]

0.11.1

• Return nil from entity when passed nil eid (issue #75)

0.11.0

• Fixed init_db in js (issue #73)
• Preliminary work for Clojure port (PR #70, #71)
• [ BREAKING ] DS now requires CLJS 0.0-3196 or later

0.10.0

• New parser for query and rules
• Added a lot of meaningful error messages and validations to query parser
• Support for _ (ignore binding) in :in and fn-clause bindings
• Fixed id allocation bug when nested maps are used for referenced entities (issue #59)
• Fixed squuids omitting leading zeros (issue #60)
• Fixed a bug when cross-referencing components caused infinite loop in Pull API (isuue #58, pull request #61)
• Pull API handles recursion more like Datomic does (issue #62)
• Exposed DB filter API to js (pull request #65)
• Added ICounted, ISeqable, IEmptyableCollection to datascript.core/DB
• Force put Datom and DB tag readers to cljs.reader/*tag-table*, do not rely on data_readers.clj

0.9.0

• Lookup refs can be inlined in query :where patterns (issue #53)
• Pull API (issue #37, pull request #51) by David Thomas Hume
• pull form in :find clause

0.8.1

• :db/unique constraints
• Upserts
• Lookup refs
• entid function

0.8.0

• Find specifications: collection :find [?e ...], tuple :find [?e ?v], and scalar :find ?e .
• Support for :db/isComponent (issue #3)
• Support for nested maps in transact! (also fixes #38)
• [ BREAKING ] Custom aggregate fns must be called via special syntax (aggregate keyword): (q '[:find (aggregate ?myfn ?e) :in $ ?myfn ...]). Built-in aggregates work as before: (q '[:find (count ?e) ...]
• Return nil from entity when passed nil eid (issue #47)
• Transaction data is now validated, with proper error messages (also fixes #48)

0.7.2

• Fixed a bug with function call after function relation was filtered out (issue #44)

0.7.1

• Fixed a bug when emtpy coll in input was not creating empty relation

0.7.0

• BTSet and its slices (returned by datoms/seek-datoms/index-range calls) now support fast reverse iteration via reverse.
• (datascript/datom e a v & [tx added]) call for creating new datoms
• Added missing aggregate funs: avg, median, variance, stddev and count-distinct (issue #42, thx @montyxcantsin)
• min and max aggregates use comparator instead of default js < comparison
• Fixed a bug when fn inside a query on empty relation returned non-empty result
• Filtered DB support via filter

0.6.0

• [ BREAKING ] Entity ids and transaction ids now have to be numbers, attributes can only be keywords (strings if used from JS)
• New method init-db for fast creation of DB from existing datoms (useful for e.g. initialization from serialized state)
• Improved performance of transact! (~20%) and initial database population (init-db, db-from-reader, ~800%)

0.5.2

• Externs provided via deps.cljs — no need to manually specify externs when using datascript dependency

0.5.1

• Ability to pass inputs (predicates/functions bindings) down to rules (issues #28 #29)

0.5.0

• Javascript version is now packaged as a proper CommonJS/RequireJS module (include via script tag still supported) (issue #39)
• Published to npm: npmjs.org/package/datascript
• [ BREAKING ] Javascript namespace is renamed from datascript.js to datascript

0.4.2

• Reference transaction id inside transaction by using :db/current-tx instead of entity id (issue #32)
• You can get id of transaction from TxReport: (get-in tx-report [:tempids :db/current-tx])
• Ability to pass arbitrary metadata along with transaction (third argument to transact!) and read it in listener callback (:tx-meta key of TxReport) (issue #32)

0.4.1

• Added :db.fn/cas transaction function (issue #20, thx @montyxcantsin)
• Added get-some, get-else, misssing? query functions (issue #21, thx @montyxcantsin)
• Fixed touch breaking entity cache (issue #17)
• Added ground fn (issue #25)
• Fixed query predicate/fn calls without free variables
• Retract incoming entity references on :db.fn/retractEntity (issue #18)
• You can now use reverse relation name when transacting entity as a map

0.4.0

Cosmetic changes to better mimic Datomic API. Useful for sharing code between Datomic and DataScript:

• Added tempid, resolve-tempid, db, transact, transact-async, index-range, squuid, squuid-time-millis
• [ BREAKING ] renamed transact to with, with to db-with

0.3.1

• Optimized speed of DB’s equiv and hash, Datom’s hash
• Entity’s touch call accessible through datascript namespace
• Accept sets in entity maps as values for :db.cardinality/many attributes

0.3.0

Proper entities implementation:

• Entities are now lazy and implement usual Map protocols
• [ BREAKING ] When accessing attribute of :db/valueType :db.type/ref, its value will be automatically expanded to entites, allowing for recursive exploration of entities graphs (e.g. (-> (d/entity db 42) :parent :parent :children))
• Entities support backwards navigation (e.g. (:person/_friends (d/entity db 42)))

0.2.1

• Externs file now can be referred as :externs [datascript/externs.js"]

0.2.0

Big performance improvements:

• New B-tree based indexes
• New set-at-a-time, hash-join powered query and rules engine
• Queries now up to 10× times faster
• You can specify DB for rule call (like ($db follows ?e1 ?e2))
• Datoms are required to have integer id and keyword attributes, but no restriction on types of values

0.1.6

• Clojure reader support (pr/read) for DB and Datom

0.1.5

• datoms and seek-datoms API calls
• referencing other entities’ tempids in transaction data (issue #10)

0.1.4

• Transactor functions via :db.fn/call (thx @thegeez)
• Vanilla JS API bindings
• [ BREAKING ] Schema keywords namespaced on a par with Datomic schema

0.1.3

• entity added