Coalesced Chunk Retrieval Protocol (CCRP) (0.1.0)

Download OpenAPI specification:Download

CCRP is a high-performance, read-only API designed to solve the N+1 request problem for chunked datasets in cloud object storage. It acts as a "byte broker" that coalesces multiple chunk requests into efficient, parallelized operations.

API landing page and conformance information

Returns information about the CCRP service, its capabilities, and conformance classes

Responses

Response samples

200

Content type

application/json

{"title": "Example CCRP Service",
"description": "CCRP service providing efficient access to chunked climate data",
"attribution": "Data provided by NOAA",
"conformsTo": ["https://ccrp.io/spec/v1/conformance/core",
"https://ccrp.io/spec/v1/conformance/range-requests"
],
"links": [{"href": "/",
"rel": "self",
"type": "application/json"
},
{"href": "/dataset",
"rel": "https://ccrp.io/spec/v1/rel/datasets",
"type": "application/json",
"title": "Available datasets"
},
{"href": "/conformance",
"rel": "conformance",
"type": "application/json",
"title": "Conformance declaration"
},
{"href": "/api",
"rel": "service-desc",
"type": "application/vnd.oai.openapi+json;version=3.0",
"title": "OpenAPI service description"
}
]
}

Conformance declaration

Returns the list of conformance classes supported by this CCRP implementation

Responses

Response samples

200

Content type

application/json

{"conformsTo": ["https://ccrp.io/spec/v1/conformance/core",
"https://ccrp.io/spec/v1/conformance/range-requests",
"https://ccrp.io/spec/v1/conformance/advanced-queries"
]
}

List available datasets

Returns a paginated list of all datasets available through this CCRP service

query Parameters

limit	integer [ 1 .. 1000 ] Default: 100 Maximum number of datasets to return
token	string Pagination token from previous response

Responses

Response samples

200

Content type

application/json

{"datasets": [{"id": "noaa-goes-17-L2",
"title": "NOAA GOES-17 Level 2 Data",
"description": "string",
"format": "zarr",
"links": [{"href": "/dataset/noaa-goes-17-L2",
"rel": "self",
"type": "application/json",
"method": "POST",
"title": "Dataset metadata",
"hreflang": "en",
"body": {"query": {"type": "object",
"properties": {"time": {"type": "string"
}
}
}
}
}
]
}
],
"links": [{"href": "/dataset",
"rel": "self",
"type": "application/json"
},
{"href": "/dataset?token=eyJsYXN0X2lkIjoibm9hYS1nb2VzLTE3LUwyIn0&limit=100",
"rel": "next",
"type": "application/json"
}
],
"nextToken": "eyJsYXN0X2lkIjoibm9hYS1nb2VzLTE3LUwyIn0"
}

Get dataset metadata

Returns detailed metadata for a specific dataset, including native format metadata

path Parameters

dataset_id

required

string

Dataset identifier

Responses

Response samples

200
404

Content type

application/json

{"id": "noaa-goes-17-L2",
"title": "NOAA GOES-17 Level 2 Data",
"description": "string",
"format": "zarr",
"latest_version": "commit-f4a2b1c8",
"native_metadata": { },
"links": [{"href": "/dataset/noaa-goes-17-L2",
"rel": "self",
"type": "application/json"
},
{"href": "/dataset/noaa-goes-17-L2/data",
"rel": "https://ccrp.io/spec/v1/rel/data",
"type": "multipart/mixed",
"title": "Query and retrieve data"
}
]
}

Plan a data query

Returns query plan information including total size and ETag for subsequent range requests

path Parameters

dataset_id

required

string

Dataset identifier

query Parameters

version	string Specific version to query (uses latest if not specified)
	object (DimensionQueries) Example: time=time[gte]=2024-01-01&time[lt]=2024-02-01&lat=lat[gte]=30&lat[lt]=40&band=C07,C08&temperature[gte]=20.0&status[nin]=error,cancelled Dimension-based query parameters

Responses

Response samples

400
404

Content type

application/json

{"error": "Invalid query syntax",
"details": "Unknown operator [foo] in dimension 'temperature[foo]'",
"dimension": "temperature"
}

Retrieve data

Retrieves actual data bytes based on query parameters. Supports HTTP Range requests for parallel downloads when used with If-Match header containing the ETag from a prior HEAD request.

Returns chunks as a multipart/mixed response where each chunk is a separate part with Content-ID header for identification.

path Parameters

dataset_id

required

string

Dataset identifier

query Parameters

version	string Specific version to query (uses latest if not specified)
	object (DimensionQueries) Example: time=time[gte]=2024-01-01&time[lt]=2024-02-01&lat=lat[gte]=30&lat[lt]=40&band=C07,C08&temperature[gte]=20.0&status[nin]=error,cancelled Dimension-based query parameters

Responses

Response samples

200
400
404
412

Content type

multipart/mixed

--boundary123
Content-ID: chunk-0-0-5-2
Content-Type: application/octet-stream
Content-Length: 32768

[binary chunk data]
--boundary123
Content-ID: chunk-0-0-5-3
Content-Type: application/octet-stream
Content-Length: 32768

[binary chunk data]
--boundary123--