AWS S3 Setup for Apple Health XML Imports

Overview

Large Apple Health XML exports upload directly to object storage (any S3-compatible bucket: AWS S3 or self-hosted) rather than streaming through the API. How a finished upload starts processing is controlled by APPLE_XML_UPLOAD_COMPLETION_MODE:

client (default) - the portal finalizes the upload and queues processing itself. You only need an S3 bucket, credentials, and a CORS policy. The SNS steps (2-5) are not required.
sns - an S3 bucket event notifies the backend through SNS. This is an AWS-only legacy path; complete every step in this guide.

If you run the default client mode - including any non-AWS S3-compatible deployment - do Step 1 and Step 6 and skip the SNS-only steps (2-5). See the Apple XML Import Guide for the upload flow itself.

Prerequisites

AWS account with appropriate permissions
AWS CLI installed (optional, but recommended)
Access to AWS Console

Step 1: Create S3 Bucket

Open S3 Console

Navigate to AWS S3 Console

Create Bucket

Click Create bucket and configure:

Bucket name: Choose a unique name (e.g., open-wearables-xml)
AWS Region: Select your preferred region (e.g., eu-north-1)
Block Public Access: Keep all boxes checked (recommended)
Bucket Versioning: Disabled (optional)
Encryption: Enable with SSE-S3 (recommended)

Create Bucket

Click Create bucket at the bottom

Using AWS CLI

aws s3api create-bucket \
  --bucket open-wearables-xml \
  --region eu-north-1 \
  --create-bucket-configuration LocationConstraint=eu-north-1

Configure Bucket CORS (required for portal uploads)

The portal uploads files directly from the browser to the bucket, which is a cross-origin request. Without a CORS policy the browser blocks the upload, and even a successful PUT hides the ETag the client needs to finalize a multipart upload. This applies to both completion modes and to any S3-compatible bucket. Apply a CORS policy that allows your portal’s origin to PUT/POST and exposes the ETag response header:

[
  {
    "AllowedOrigins": ["https://your-portal-domain.com"],
    "AllowedMethods": ["PUT", "POST"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }
]

Using AWS CLI

aws s3api put-bucket-cors \
  --bucket open-wearables-xml \
  --cors-configuration file://cors.json

Replace https://your-portal-domain.com with your frontend origin (for local development, http://localhost:3000). The ExposeHeaders: ["ETag"] entry is mandatory for multipart uploads - without it the client cannot read part ETags and completion fails.

CORS is not an access-control mechanism. It only relaxes the browser’s same-origin rule for reading cross-origin responses; it grants no access to the bucket. Every upload still requires a short-lived, backend-issued presigned URL, and the bucket keeps Block Public Access on. Scope AllowedOrigins to your own portal domain (avoid *) as defense in depth, and keep AllowedMethods to only PUT/POST.

Steps 2-5 apply only to sns completion mode. Running the default client mode? Skip ahead to Step 6 - you need the bucket above, its CORS policy, and credentials, but no SNS topic, subscription, or S3 event notification.

Open SNS Console

Navigate to AWS SNS Console

Create Topic

Click Topics → Create topic and configure:

Type: Standard
Name: e.g., owear

Create Topic

Click Create topic at the bottom

Copy Topic ARN

After creation, copy the ARN from the details page. It will look like:

arn:aws:sns:eu-north-1:123456789012:owear

You’ll need this for the environment variables and access policy.

Using AWS CLI

aws sns create-topic \
  --name owear \
  --region eu-north-1

# Note the TopicArn from the output

The SNS topic needs permission to receive publish events from S3.

Open Topic Access Policy

In the SNS console, click on your topic → Edit → expand Access policy

Add S3 Publish Permission

Add the following statement to the policy’s Statement array (update the ARNs with your values):

{
  "Sid": "AllowS3Publish",
  "Effect": "Allow",
  "Principal": {
    "Service": "s3.amazonaws.com"
  },
  "Action": "SNS:Publish",
  "Resource": "arn:aws:sns:eu-north-1:123456789012:owear",
  "Condition": {
    "ArnLike": {
      "aws:SourceArn": "arn:aws:s3:::open-wearables-xml"
    }
  }
}

Save Changes

Click Save changes to apply the updated policy

Replace 123456789012 with your AWS Account ID, open-wearables-xml with your bucket name, and owear with your topic name.

Configure S3 to send notifications to the SNS topic when files are uploaded.

Open Bucket Properties

Go to your S3 bucket → Properties tab

Create Event Notification

Scroll to Event notifications → Click Create event notificationConfigure:

Name: e.g., xml-upload-notification
Event types: Check All object create events (or specifically s3:ObjectCreated:Post)
Suffix: .xml (optional, to only notify for XML files)
Destination: Select SNS topic
SNS topic: Select your topic (e.g., owear)

Save Configuration

Click Save changes

Using AWS CLI

cat > notification.json <<EOF
{
  "TopicConfigurations": [
    {
      "TopicArn": "arn:aws:sns:eu-north-1:123456789012:owear",
      "Events": ["s3:ObjectCreated:*"],
      "Filter": {
        "Key": {
          "FilterRules": [
            {"Name": "suffix", "Value": ".xml"}
          ]
        }
      }
    }
  ]
}
EOF

aws s3api put-bucket-notification-configuration \
  --bucket open-wearables-xml \
  --notification-configuration file://notification.json

Create an HTTPS subscription so SNS delivers notifications to your backend.

Open Topic Subscriptions

In the SNS console, click on your topic → Create subscription

Configure Subscription

Topic ARN: Select your topic (e.g., owear)
Protocol: HTTPS
Endpoint: Your backend’s SNS notification URL, e.g.:

https://your-backend-domain.com/api/v1/sns/notification

Create Subscription

Click Create subscription. AWS will send a SubscriptionConfirmation request to your endpoint. The backend handles this automatically and confirms the subscription.

Verify Confirmation

After a few seconds, refresh the subscriptions page. The status should change from Pending confirmation to Confirmed.

The backend must be publicly accessible for SNS to reach it. If developing locally, use a tool like ngrok to expose your local server.

Step 6: Update Environment Variables

Add the AWS configuration to your .env file:

# AWS S3 Configuration
AWS_BUCKET_NAME=open-wearables-xml
AWS_REGION=eu-north-1
AWS_ACCESS_KEY_ID=your-access-key-id
AWS_SECRET_ACCESS_KEY=your-secret-access-key

# SNS Configuration (SNS completion mode only)
AWS_SNS_TOPIC_ARN=arn:aws:sns:eu-north-1:123456789012:owear

The Celery worker also needs these AWS variables. The import task runs on the worker and downloads the uploaded XML directly from the bucket (get_object), so in a multi-service deployment (Docker Compose, Railway, etc.) the worker service needs the same AWS_BUCKET_NAME, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION (plus any AWS_ENDPOINT_URL) as the backend. Without them the task fails with RuntimeError: S3 client not configured. This applies in both completion modes.

Optional: S3-compatible storage and upload tuning

These variables are all optional; their defaults target real AWS S3. Set the endpoint variables to run the same upload flow against a self-hosted S3-compatible server, and the rest to tune upload behavior.

Variable	Default	Purpose
`AWS_ENDPOINT_URL`	(unset - real AWS)	Custom S3-compatible endpoint (e.g. `http://storage:9000`). When set, the shared client factory forces path-style addressing + SigV4 so presigned and multipart uploads behave consistently across any S3-compatible bucket. Used by every backend-side S3 call.
`AWS_PUBLIC_ENDPOINT_URL`	(falls back to `AWS_ENDPOINT_URL`)	Browser-facing endpoint for presigned POST + multipart part URLs. The backend reaches storage at `AWS_ENDPOINT_URL` (e.g. a Docker service name the browser can’t resolve); point this at the port-mapped host instead (e.g. `http://localhost:9000`) to avoid editing `/etc/hosts`. Not needed on real AWS.
`APPLE_XML_UPLOAD_COMPLETION_MODE`	`client`	How a finished upload triggers import: `client` dispatches from the `/complete` call (any S3-compatible bucket); `sns` waits for an S3 bucket event via SNS (AWS only, as configured above). Only the selected mode dispatches, so a file is never processed twice.
`APPLE_XML_MAX_FILE_SIZE_BYTES`	`5368709120` (5 GiB)	Product-level upload cap (accepts 5 MiB - 5 TiB). The portal mirrors this default for client-side validation.
`APPLE_XML_MULTIPART_PART_SIZE_BYTES`	`104857600` (100 MiB)	Recommended browser chunk size returned by the multipart `create` call (accepts 5 MiB - 5 GiB). Clients should use the returned `part_size`, not hard-code it.

Browser uploads also require the bucket CORS policy configured above.

How to Get AWS Credentials

Go to IAM Console
Click Users → Add users
User name: open-wearables-app
Select Access key - Programmatic access
Click Next: Permissions
Attach policies:
- AmazonS3FullAccess (or create a custom policy with specific bucket access)
- AmazonSNSFullAccess (SNS mode only, or create a custom policy with specific topic access). A custom SNS policy must include SNS:ConfirmSubscription - the backend calls it to confirm the HTTPS subscription; without it the subscription stays on “Pending confirmation”.
Click through to create the user
Copy the Access key ID and Secret access key immediately (you won’t see the secret again)

Keep your AWS credentials secure! Never commit them to version control.

Step 7: Restart Services

After updating the .env file, restart your services:

# Using Docker Compose
docker compose down
docker compose up -d

Verify Setup

Test your configuration:

Check SNS Subscription (SNS mode only)

In the SNS console, verify your subscription status is Confirmed.

Test Upload

Upload a small XML file through the portal, or use the multipart upload script for manual testing. See the Apple XML Import Guide for the flow and example scripts.

Check Backend Logs

After uploading, check the backend logs for processing:

docker compose logs -f app

You should see logs about dispatching process_aws_upload tasks - in client mode these are queued from the /complete call, in sns mode from the incoming SNS notification.

Troubleshooting

Notifications Not Triggering Processing

Possible causes:

S3 event notification not configured correctly
SNS topic access policy doesn’t allow S3 to publish
Wrong bucket or topic ARN in configuration

Fix:

Verify the event notification in S3 bucket properties
Check the SNS access policy allows s3.amazonaws.com to publish
Test by manually uploading a file to S3 and checking backend logs

Access Denied Errors

Possible causes:

IAM user doesn’t have required permissions
Wrong AWS credentials in .env
Bucket or topic in different region

Fix:

Verify IAM user has S3 and SNS permissions
Double-check AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
Ensure AWS_REGION matches where your bucket and topic are located

Import fails with 'S3 client not configured'

Cause: the Celery worker service is missing AWS credentials. The import task runs on the worker and reads the file from the bucket, so the worker needs its own AWS configuration - not just the backend/API container.Fix:

Set AWS_BUCKET_NAME, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION (plus any AWS_ENDPOINT_URL) on the worker service, then restart it
In Docker Compose, confirm the worker service loads the same env file as the backend

Browser upload fails with 'Load Failed' or a CORS error

Cause: the bucket has no CORS policy, or it does not expose the ETag header. Direct-from-browser uploads are cross-origin, so the preflight is rejected (or the client cannot read part ETags to complete a multipart upload).Fix:

Apply the bucket CORS policy: allow your portal origin with PUT/POST and set ExposeHeaders: ["ETag"]
Confirm AllowedOrigins matches the exact scheme + host + port the portal is served from

Cost Optimization

S3 Storage

Uploaded XML files remain in S3 for your records
Set up Lifecycle Rules to archive or delete old files if needed
Consider moving to Glacier for long-term archival

SNS Notifications

SNS free tier includes 100,000 HTTP/HTTPS notifications per month
Check AWS SNS pricing for current limits and rates beyond free tier

Data Transfer

Keep S3 bucket and servers in the same AWS region
Use presigned URLs to avoid routing data through your server

Request Costs

S3 and SNS costs are negligible for typical usage
See AWS S3 pricing for current rates

Security Best Practices

Use IAM Roles: If running on EC2/ECS, use IAM roles instead of access keys
Restrict Bucket Access: Only allow specific IAM users/roles to access the bucket
Enable Encryption: Use SSE-S3 or SSE-KMS for data at rest
Monitor Access: Enable CloudTrail logging for S3 and SNS
Rotate Credentials: Regularly rotate AWS access keys

Apple XML Import Guide - Using the presigned URL endpoint
How to Add a Provider - Extending the platform
Integration Guide - Testing your setup

Getting started

Architecture

Health Scores

Providers

Open Wearables Mobile App

Sync SDK

MCP Server

Developer Guides

Configuration

Deployment

Support

AWS S3 Setup for Apple Health XML Imports

Overview

Prerequisites

Step 1: Create S3 Bucket

Configure Bucket CORS (required for portal uploads)

Step 6: Update Environment Variables

Optional: S3-compatible storage and upload tuning

Step 7: Restart Services

Verify Setup

Troubleshooting

Cost Optimization

S3 Storage

SNS Notifications

Data Transfer

Request Costs

Security Best Practices

​Overview

​Prerequisites

​Step 1: Create S3 Bucket

​Configure Bucket CORS (required for portal uploads)

​Step 2: Create SNS Topic (SNS mode only)

​Step 3: Update SNS Topic Access Policy (SNS mode only)

​Step 4: Add S3 Event Notification (SNS mode only)

​Step 5: Create SNS Subscription (SNS mode only)

​Step 6: Update Environment Variables

​Optional: S3-compatible storage and upload tuning

​Create IAM User with S3 and SNS Permissions

​Step 7: Restart Services

​Verify Setup

​Troubleshooting

​Cost Optimization

S3 Storage

SNS Notifications

Data Transfer

Request Costs

​Security Best Practices

​Related Guides

Overview

Prerequisites

Step 1: Create S3 Bucket

Configure Bucket CORS (required for portal uploads)

Step 2: Create SNS Topic (SNS mode only)

Step 3: Update SNS Topic Access Policy (SNS mode only)

Step 4: Add S3 Event Notification (SNS mode only)

Step 5: Create SNS Subscription (SNS mode only)

Step 6: Update Environment Variables

Optional: S3-compatible storage and upload tuning

Create IAM User with S3 and SNS Permissions

Step 7: Restart Services

Verify Setup

Troubleshooting

Cost Optimization

Security Best Practices

Related Guides