You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/sync/api/config.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -566,6 +566,8 @@ If provided must be one of `MEMORY` or `FAST_FILE`.
566
566
567
567
Path to root folder for storing data on the filesystem.
568
568
569
+
This must be persistent storage that survives restarts. See [Optimizing for disk](/docs/sync/guides/deployment#optimizing-for-disk) in the deployment guide for what to run it on.
Copy file name to clipboardExpand all lines: website/docs/sync/guides/deployment.md
+6-2Lines changed: 6 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -129,19 +129,23 @@ You can deploy it anywhere you can run a container with a filesystem and exposed
129
129
-[Fly.io](/docs/sync/integrations/fly)
130
130
-[Render](/docs/sync/integrations/render)
131
131
132
+
The [AWS guide](/docs/sync/integrations/aws#deploy-electric) goes into the most depth, covering how to choose an ECS launch type and storage backing. It comes with [example Terraform and Pulumi configurations](https://github.com/electric-sql/electric-aws) that provision the whole stack — database, sync service and load balancer.
133
+
132
134
### Docker container
133
135
134
136
Images are deployed to Docker Hub at [electricsql/electric](https://hub.docker.com/r/electricsql/electric).
135
137
136
138
### Optimizing for disk
137
139
138
-
Electric caches [Shape logs](/docs/sync/api/http#shape-log) and metadata on the filesystem. Your Electric host must provide a persistent filesystem. Ideally this should be large, fast and locally mounted, such as a NVMe SSD. If you're configuring a machine and you want to optimise it for Electric, the factors to optimise for, in order of important, are:
140
+
Electric caches [Shape logs](/docs/sync/api/http#shape-log) and metadata on the filesystem. Your Electric host must provide a persistent filesystem. Ideally this should be large, fast and locally mounted, such as a NVMe SSD. If you're configuring a machine and you want to optimise it for Electric, the factors to optimise for, in order of importance, are:
139
141
140
142
1. disk speed — low latency, high throughput reads and writes
141
143
2. memory
142
144
3. CPU
143
145
144
-
For example, on AWS, [Storage Optimized](https://aws.amazon.com/ec2/instance-types/#Storage_Optimized) instances such as the `i3en.large`, or on Hetzner the [SX-line](https://www.hetzner.com/dedicated-rootserver/matrix-sx/) of dedicated servers would both be great choices.
146
+
For example, on AWS, [storage optimized](https://aws.amazon.com/ec2/instance-types/#Storage_Optimized) instances such as the `i4i` family, or general purpose instances that ship local NVMe such as `m6id`, are both good choices. On Hetzner, the [SX-line](https://www.hetzner.com/dedicated-rootserver/matrix-sx/) of dedicated servers works well.
147
+
148
+
For a worked example, see [ECS on EC2](/docs/sync/integrations/aws#ecs-on-ec2) in the AWS guide. It covers choosing between local NVMe and an attached EBS volume, and [preparing the disk on boot](/docs/sync/integrations/aws#bootstrapping-the-data-disk).
Copy file name to clipboardExpand all lines: website/docs/sync/guides/upgrading.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -226,6 +226,8 @@ With `maxSurge: 1` and `maxUnavailable: 0`, Kubernetes will:
226
226
227
227
This example uses EC2 launch type with a host bind mount for shared storage. Both old and new tasks share the same directory on the EC2 host.
228
228
229
+
See [ECS on EC2](/docs/sync/integrations/aws#ecs-on-ec2) in the AWS guide for how to provision this. The [example configurations](https://github.com/electric-sql/electric-aws) run a single container instance per cluster, which satisfies the placement requirement below.
230
+
229
231
> [!Warning] Same-host placement
230
232
> ECS does not guarantee that the new task lands on the same host as the old one. To ensure both tasks share the same host volume, your ECS cluster must have exactly one EC2 instance matching your placement constraint, or use a custom instance attribute to pin tasks to a specific host.
Copy file name to clipboardExpand all lines: website/docs/sync/integrations/aws.md
+63-5Lines changed: 63 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,16 +39,74 @@ GRANT rds_replication TO someuser;
39
39
40
40
### Deploy Electric
41
41
42
-
AWS provides a [wide range of container hosting](https://aws.amazon.com/containers). For example, you can deploy Electric to [AWS Elastic Container Service](https://aws.amazon.com/efs) using [AWS Fargate](https://aws.amazon.com/fargate).
42
+
The recommended way to run Electric on AWS is [Elastic Container Service](https://aws.amazon.com/ecs) (ECS). We maintain example [Terraform and Pulumi configs](#examples) that stand up everything described below.
43
43
44
-
You should store Shape logs to a persistent disk (not an ephemoral filesystem). For example using [Amazon Elastic File System](https://aws.amazon.com/efs).
44
+
Electric caches [shape logs](/docs/sync/api/http#shape-log) on disk and treats them as a rebuildable cache: if the disk is lost, Electric rebuilds from Postgres. Disk speed directly drives sync performance, so the main deployment decision is where that disk lives. There are three good options on ECS:
45
+
46
+
| Launch type | Storage | Characteristics |
47
+
|---|---|---|
48
+
|[Fargate](https://aws.amazon.com/fargate/)| Ephemeral task storage | Simplest; cache is lost on every deploy |
49
+
| EC2, storage‑optimized instance (`i4i`, `m6id`) | Local NVMe instance store | Fastest possible disk; survives task restarts, not host replacement |
50
+
| EC2, compute instance (`m6a`, `m6i`, `m7a`, `m7i`) | Attached gp3 EBS data volume | Cheaper; IOPS and throughput independently tunable, in place |
51
+
52
+
Fargate is fine for evaluation and light workloads. For production workloads, EC2 gives you dramatically better disk for the money — this is how [Electric Cloud](/cloud/) runs.
53
+
54
+
#### ECS on EC2
55
+
56
+
The pattern our example configs implement is one task per host:
57
+
58
+
- A **launch template** using the ECS-optimized Amazon Linux 2023 AMI, with a user-data boot script (below) that prepares the data disk and joins the instance to the cluster.
59
+
- An **auto scaling group** (min = max = 1) with scale-in protection and a termination lifecycle hook, so ECS can drain tasks before an instance is replaced.
60
+
- An **ECS capacity provider** with managed scaling and managed termination protection, which lets ECS drive the ASG rather than you managing instances directly.
61
+
- A **task definition** sized to the whole host: all of its vCPUs, and its memory minus ~2 GiB for the OS, Docker and ECS agent. (Undersize this reservation and task placement fails with `RESOURCE:MEMORY` errors.) The task uses `awsvpc` networking behind an ALB targeting `/v1/health`.
62
+
63
+
#### Bootstrapping the data disk
64
+
65
+
Both storage options — local NVMe instance store and attached gp3 EBS — appear as non-root `/dev/nvme*` devices on Nitro instances, so a single boot script handles either. On first boot it:
66
+
67
+
1. finds every non-root NVMe device, waiting up to 60s (EBS volumes attach asynchronously; instance store is present immediately)
68
+
2. RAID0s them with `mdadm` if there's more than one (larger `i4i`/`m6id` sizes ship multiple disks)
69
+
3. formats XFS and mounts at `/mnt/nvme` with `noatime`, by filesystem UUID rather than device path (NVMe enumeration order isn't guaranteed, and a reassembled RAID array can come back under a different name)
70
+
4. creates a data directory owned by uid 1000 (the Electric container user) and writes the ECS cluster-join config
71
+
72
+
The task then bind-mounts that directory and sets [`ELECTRIC_STORAGE_DIR`](/docs/sync/api/config#electric-storage-dir) to it. The full script is [`shared/user-data.sh.tpl`](https://github.com/electric-sql/electric-aws/blob/main/shared/user-data.sh.tpl) in the examples repo; its core is:
73
+
74
+
```bash
75
+
# Discover non-root NVMe devices (instance store or EBS — both
For **gp3 EBS**, size, IOPS and throughput are independent knobs you can raise later without downtime (`terraform apply` / `pulumi up` modifies the volume in place; size can grow but never shrink). The 3,000 IOPS / 125 MB/s baseline is free and roughly matches Fargate ephemeral storage; scale up if you see IO wait on shape recomputes or cold starts.
101
+
102
+
For **NVMe instance store**, capacity comes with the instance type — e.g. 118 GB on an `m6id.large`, 468 GB on an `i4i.large`, with `i4i` giving ~4x the storage per vCPU. You can't tune it, but it's far faster than any networked volume.
103
+
104
+
Either way, remember the disk contents don't need to survive host replacement — Electric re-syncs from Postgres — but replacing a host does mean re-syncing, so prefer in-place tuning over instance-type churn once you're in production.
45
105
46
106
### Deploy your app
47
107
48
108
AWS provides a range of [website hosting options](https://aws.amazon.com/getting-started/hands-on/host-static-website/). For example you can deploy a static app to [AWS Amplify](https://aws.amazon.com/amplify).
49
109
50
110
## Examples
51
111
52
-
### AWS Terraform
53
-
54
-
We have an example Terraform repo at [electric-sql/terraform-aws](https://github.com/electric-sql/terraform-aws).
112
+
The [electric-sql/electric-aws](https://github.com/electric-sql/electric-aws) repo contains equivalent [Terraform](https://github.com/electric-sql/electric-aws/tree/main/terraform) and [Pulumi](https://github.com/electric-sql/electric-aws/tree/main/pulumi) configs implementing everything on this page — VPC, RDS with logical replication, ECS (Fargate or EC2 with either storage option) and an ALB.
0 commit comments