-
Notifications
You must be signed in to change notification settings - Fork 201
/
Copy pathelectric-api.yaml
479 lines (437 loc) · 19.5 KB
/
electric-api.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
openapi: 3.1.0
info:
title: Electric HTTP API
description: |-
[HTTP API](https://electric-sql.com/docs/api/http) to sync
partial replicas of your Postgres data into local apps
and services.
See the [Electric documentation](https://electric-sql.com/docs/intro)
for more information.
# FIXME: set version dynamically
# version: 0.0.1
servers:
- url: http://localhost:3000
description: Local server
paths:
/v1/shape:
get:
summary: Get Shape
description: |-
Load the initial data for a shape and poll for real-time updates.
Define your shape using the `table` and `where` parameters.
Use `offset` to fetch data from a specific position in the shape
log and the `live` parameter to consume real-time updates.
parameters:
# Query parameters
- name: table
in: query
schema:
type: string
examples:
simple:
value: issues
summary: the issues table in the public schema
qualified:
value: foo.issues
summary: the issues table in the foo schema
required: true
description: |-
Root table of the shape. Must match a table in your Postgres database.
Can be just a tablename, or can be prefixed by the database schema
using a `.` delimiter, such as `foo.issues`. If you don't provide
a schema prefix, then the table is assumed to be in the `public.` schema.
# Query parameters
- name: offset
in: query
schema:
type: string
examples:
initial_sync:
value: "-1"
summary: sync the shape from the start
ongoing_sync:
value: "26800584_4"
summary: continue syncing from offset `26800584_4`
required: true
description: |-
The offset in the shape stream. This is like a cursor that specifies
the position in the shape log to request data from.
When making an initial request to sync a shape from scratch, you
**must** set the `offset` to `-1`. Then, when continuing to
sync data, you should set the `offset` to the last offset you have
already recieved, to continue syncing new data from that position
in the stream.
Note that when `offset` is not `-1` then you must also provide
the shape's `handle`.
- name: live
in: query
schema:
type: boolean
description: |-
Whether to wait for live updates or not.
When the `live` parameter is omitted or set to `false`, the server
will always return immediately, with any data it has, followed by an
up-to-date message.
Once you're up-to-date, you should set the `live` parameter to `true`.
This puts the server into live mode, where it will hold open the
connection, waiting for new data arrive.
This allows you to implement a long-polling strategy to consume
real-time updates.
- name: cursor
in: query
schema:
type: string
description: |-
This is a cursor generated by the server during live requests. It helps bust caches for
responses from previous long-polls.
- name: handle
in: query
schema:
type: string
example: "3833821-1721812114261"
description: |-
The shape handle returned by the initial shape request.
This is a required parameter when this is not an initial sync request,
i.e. when offset is not `-1`.
- name: where
in: query
schema:
type: string
description: |-
Optional where clause to filter rows in the `table`.
This should be a valid PostgreSQL WHERE clause using SQL syntax.
For more details on what is supported and what is optimal,
see the [where clause documentation](https://electric-sql.com/docs/guides/shapes#where-clause).
examples:
title_filter:
value: '"title=''Electric''"'
summary: Only include rows where the title is 'Electric'.
status_filter:
value: '"status IN (''backlog'', ''todo'')"'
summary: Only include rows whose status is either 'backlog' or 'todo'.
- name: columns
in: query
schema:
type: string
description: |-
Optional list of columns to include in the rows from the `table`.
They should always include the primary key columns, and should be formed
as a comma separated list of column names exactly as they are in the database schema.
If the identifier was defined as case sensitive and/or with special characters, then\
you must quote it in the `columns` parameter as well.
examples:
select_columns:
value: "id,title,status"
summary: Only include the id, title, and status columns.
select_columns_special:
value: 'id,"Status-Check"'
summary: Only include id and Status-Check columns, quoting the identifiers where necessary.
- name: replica
in: query
schema:
type: string
enum:
- default
- all
description: |-
Modifies the data sent in update and delete change messages.
When `replica=default` (the default) only changed columns are
included in the `value` of an update message and only the primary
keys are sent for a delete.
When set to `full` the entire row will be sent for updates and
deletes.
Note that insert operations always include the full row,
in either mode.
# Headers
- name: If-None-Match
in: header
schema:
type: string
# TODO: is this description below correct?
description: Re-validate the shape if the etag doesn't match.
responses:
"200":
description: The shape request was successful.
headers:
cache-control:
schema:
type: string
example: "public, max-age=60, stale-while-revalidate=300"
description: |-
Cache control header as a string of comma separated directives.
Supported directives are: `max-age`, `stale-while-revalidate`.
etag:
schema:
type: string
example: "3833821-1721812114261:26800584_4:26800584_4"
description: |-
Etag header specifying the shape handle and offset for efficient caching.
In the format `{shape_handle}:{start_offset}:{end_offset}`.
electric-cursor:
schema:
type: string
example: "1674440"
description: |-
If present, provides a cursor to use as the value of the `cursor`
parameter in the next `live` mode long polling request.
This works around some inconsistent request coalescing behaviour
with different CDNs.
electric-offset:
schema:
type: string
example: "26800584_4"
description: |-
The latest offset in the batch of data you have recieved.
If no data is returned, this will be equal to the `offset` parameter
you have provided.
Must be used as the value of the `offset` parameter in your
next request.
electric-handle:
schema:
type: string
example: "3833821-1721812114261"
description: |-
The shape handle.
Must be provided as the value of the `handle` parameter when making
subsequent requests where `offset` is not `-1`.
electric-schema:
schema:
type: string
example: '{"id":{"type":"int4","dimensions":0},"title":{"type":"text","dimensions":0},"status":{"type":"text","dimensions":0,"max_length":8}}'
description: |-
Only present on responses to non-live requests.
A JSON string of an object that maps column names to the corresponding schema object.
The schema object contains the type of the column, the number of dimensions, and possibly additional properties.
Non-array types have a dimension of `0`, while array types have a dimension of 1 or more.
For instance, an array of booleans would have a type of `bool` and a dimension of `1`.
Some types provide additional properties,
e.g.: `VARCHAR(8)` has an additional `“max_length": 8` property,
`BPCHAR(9)` has an additional `"length": 9` property,
`TIME(3)` has an additional `"precision": 3` property,
`NUMERIC(8,5)` has additional `"precision": 8` and `"scale": 5` properties,
`INTERVAL(4)` has an additional `"precision": 4` property,
`INTERVAL MINUTE TO SECOND` has an additional `"fields": "MINUTE TO SECOND"` property,
`BIT(5)` has an additional `"length": 5` property.
electric-up-to-date:
schema:
description: |-
If present, this header indicates that the response ends with
an `up-to-date` control message, indicating that the client has
recieved all of the data that the server is aware of and can
safely process/apply any accumulated messages.
content:
application/json:
schema:
type: array
description: Array of message objects
items:
type: object
description: Message object
properties:
headers:
type: object
description: |-
Metadata about the message.
Messages can be `control` messages, providing information or
instructions to the client. Or they can be operations that
performed a certain `operation` on a row of data in the shape.
properties:
control:
type: string
enum:
- up-to-date
- must-refetch
operation:
type: string
enum:
- insert
- update
- delete
description: The type of operation that is performed on the row of the shape that is identified by the `key`.
lsn:
type: string
description: |-
The logical sequence number of the operation.
Only present on operations that were received from the event stream.
It's missing on initial query results and on compacted items.
Operations with the same LSN were committed in the same transaction and
can be ordered by `op_position` within the same LSN.
op_position:
type: string
description: |-
The position of the operation in the transaction.
Only present on operations that were received from the event stream.
It's missing on initial query results and on compacted items.
last:
type: boolean
description: |-
Whether this is the last operation in the transaction for this shape.
Last operation in a transaction for the shape does not mean a last
operation in the transaction for the database.
Only present on operations that were received from the event stream.
It's missing on initial query results and on compacted items.
txids:
type: array
description: |-
The list of transaction IDs that this operation was part of.
Currently, this will only contain a single transaction ID, but future
stream processors may merge operations from multiple transactions into a single
operation in the event stream.
key:
type: string
description: Row ID
value:
type: object
description: |-
The row data.
Note that this does not necessarily contain the whole row:
- for inserts it will contain the whole row
- for updates it will contain the primary key and the changed values
- for deletes it will contain just the primary key
The values are strings that are formatted according to Postgres' display settings.
Some Postgres types support several display settings, we format values consistently according to the following display settings:
- `bytea_output = 'hex'`
- `DateStyle = 'ISO, DMY'`
- `TimeZone = 'UTC'`
- `IntervalStyle = 'iso_8601'`
- `extra_float_digits = 1`
example:
- headers:
operation: insert
lsn: 1234
op_position: 0
key: issue-1
value:
id: issue-1
title: Electric
status: backlog
- headers:
operation: insert
lsn: 1234
op_position: 7
key: issue-2
value:
id: issue-2
title: Hello
status: backlog
- headers:
control: up-to-date
"204":
description: >-
No content. The `live=true` polling request timed out without
any new content to process.
headers:
electric-cursor:
schema:
type: string
example: "1674440"
description: |-
Provides a cursor to use as the value of the `cursor` parameter
in the next long polling request.
This works around some inconsistent request coalescing behaviour
with different CDNs.
"400":
description: Bad request.
"404":
description: Database not found.
"409":
description:
The requested offset for the given shape no longer exists.
Client should sync the shape using the relative path from the location header.
headers:
location:
schema:
type: string
description: Relative path for syncing the latest version of the requested shape.
electric-handle:
schema:
type: string
description: Handle of the new shape that must be used in client requests from now on.
content:
application/json:
schema:
type: array
description: Array of message objects
items:
type: object
description: Message object
properties:
headers:
type: object
description: |-
Metadata describing the control message.
The `control` message returned will be a `must-refetch` message,
which a client should detect and throw away any local data and
re-sync from scratch using the new shape handle available in the
`electric-handle` header of the response.
properties:
control:
type: "string"
enum:
- must-refetch
example:
- headers:
control: must-refetch
"429":
description:
Too many requests. The server is busy with other requests, potentially
because of high contention on the underlying database. Retry after a little time.
content:
application/json:
schema:
type: object
properties:
message:
type: string
description: Error message
example:
message: "Could not establish connection to database - try again later"
delete:
summary: Delete Shape
description: |-
Deletes the shape from the Electric sync engine.
This clears the shape log and forces any clients requesting the shape
to create a new shape and resync from scratch.
**NOTE** Delete shape only works if Electric is configured to `allow_shape_deletion`.
parameters:
# Query parameters
- name: table
in: query
schema:
type: string
examples:
simple:
value: issues
summary: the issues table in the public schema
qualified:
value: foo.issues
summary: the issues table in the foo schema
required: true
description: |-
The name of the table for which to delete the shape.
Can be qualified by the schema name.
# Query parameters
- name: source_id
in: query
schema:
type: string
description: |-
The ID of the database from which to delete the shape.
This is required only if Electric manages several databases.
- name: handle
in: query
schema:
type: string
example: "3833821-1721812114261"
description:
Optional, deletes the current shape if it matches the `handle` provided.
If not provided, deletes the current shape.
responses:
"202":
description: |-
Accepted. The shape has been deleted (or to be more precise: the shape handle
has been invalidated and the storage will be cleaned up eventually).
"400":
description: Bad request.
"404":
description: Database or shape not found (or shape deletion is not enabled).