Skip to content

dedup4j

Store identical bytes once — deduplicated blob storage for Spring Boot, backed by your own database and object store.

Not yet on Maven Central

dedup4j 0.1.0 has not been published. These coordinates are the intended release coordinates and cannot be resolved from Maven Central today. The Maven Central namespace is confirmed in release gate G0 and this file is the single place it is edited.

When two users upload the same file, dedup4j recognises the content, stores the bytes once, and hands back a reference to the single stored copy. Your application keeps its own asset records; dedup4j owns the physical content and counts who still needs it.

Group ID com.edem
Version 0.1.0-SNAPSHOT
Java 21 or later
Spring Boot built and tested against 4.1.1
License Apache License 2.0

Add the dependency

<dependency>
    <groupId>com.edem</groupId>
    <artifactId>dedup4j-spring-boot-starter</artifactId>
    <version>0.1.0-SNAPSHOT</version>
</dependency>
implementation 'com.edem:dedup4j-spring-boot-starter:0.1.0-SNAPSHOT'

The starter auto-configures everything against your existing DataSource. It does not open its own database connection pool or require a separate datastore.

Quick start Installation

Two things dedup4j does not do

Read this before designing around it

It does not generate presigned URLs. dedup4j hands you a BlobReference and can stream content back to you, but issuing time-limited direct-to-storage URLs is your application's job. See Retrieval, retain & release.

It cannot make the database write and the object-store write atomic. Two systems, no shared transaction. dedup4j is explicit about the ordering it chooses and what reconciliation exists for the gap. See Architecture & limitations.

These are the two expectations most likely to be formed and then disappointed, so they are stated before the installation instructions rather than after.

Where to go next

If you want to Read
Add it to a project Installation
See it work end to end Quick start
Choose local, S3, or Azure Storage providers
Understand reference counting Retrieval, retain & release
Know what it cannot promise Architecture & limitations

Source on GitHub · Contributing