Skip to main content

​
Overview

To enable large XML file uploads via presigned URLs, you need to configure AWS S3 and SNS. The frontend handles uploads through presigned URLs, and SNS notifications trigger automatic processing when files land in S3. This guide walks through creating and configuring the required AWS resources.

​
Prerequisites

  • AWS account with appropriate permissions
  • AWS CLI installed (optional, but recommended)
  • Access to AWS Console

​
Step 1: Create S3 Bucket

1

Open S3 Console

Navigate to AWS S3 Console
2

Create Bucket

Click Create bucket and configure:
  • Bucket name: Choose a unique name (e.g., open-wearables-xml)
  • AWS Region: Select your preferred region (e.g., eu-north-1)
  • Block Public Access: Keep all boxes checked (recommended)
  • Bucket Versioning: Disabled (optional)
  • Encryption: Enable with SSE-S3 (recommended)
3

Create Bucket

Click Create bucket at the bottom
aws s3api create-bucket \
  --bucket open-wearables-xml \
  --region eu-north-1 \
  --create-bucket-configuration LocationConstraint=eu-north-1

​
Step 2: Create SNS Topic

1

Open SNS Console

Navigate to AWS SNS Console
2

Create Topic

Click Topics → Create topic and configure:
  • Type: Standard
  • Name: e.g., owear
3

Create Topic

Click Create topic at the bottom
4

Copy Topic ARN

After creation, copy the ARN from the details page. It will look like:
arn:aws:sns:eu-north-1:123456789012:owear
You’ll need this for the environment variables and access policy.
aws sns create-topic \
  --name owear \
  --region eu-north-1

# Note the TopicArn from the output

​
Step 3: Update SNS Topic Access Policy

The SNS topic needs permission to receive publish events from S3.
1

Open Topic Access Policy

In the SNS console, click on your topic → Edit → expand Access policy
2

Add S3 Publish Permission

Add the following statement to the policy’s Statement array (update the ARNs with your values):
{
  "Sid": "AllowS3Publish",
  "Effect": "Allow",
  "Principal": {
    "Service": "s3.amazonaws.com"
  },
  "Action": "SNS:Publish",
  "Resource": "arn:aws:sns:eu-north-1:123456789012:owear",
  "Condition": {
    "ArnLike": {
      "aws:SourceArn": "arn:aws:s3:::open-wearables-xml"
    }
  }
}
3

Save Changes

Click Save changes to apply the updated policy
Replace 123456789012 with your AWS Account ID, open-wearables-xml with your bucket name, and owear with your topic name.

​
Step 4: Add S3 Event Notification

Configure S3 to send notifications to the SNS topic when files are uploaded.
1

Open Bucket Properties

Go to your S3 bucket → Properties tab
2

Create Event Notification

Scroll to Event notifications → Click Create event notificationConfigure:
  • Name: e.g., xml-upload-notification
  • Event types: Check All object create events (or specifically s3:ObjectCreated:Post)
  • Suffix: .xml (optional, to only notify for XML files)
  • Destination: Select SNS topic
  • SNS topic: Select your topic (e.g., owear)
3

Save Configuration

Click Save changes
cat > notification.json <<EOF
{
  "TopicConfigurations": [
    {
      "TopicArn": "arn:aws:sns:eu-north-1:123456789012:owear",
      "Events": ["s3:ObjectCreated:*"],
      "Filter": {
        "Key": {
          "FilterRules": [
            {"Name": "suffix", "Value": ".xml"}
          ]
        }
      }
    }
  ]
}
EOF

aws s3api put-bucket-notification-configuration \
  --bucket open-wearables-xml \
  --notification-configuration file://notification.json

​
Step 5: Create SNS Subscription

Create an HTTPS subscription so SNS delivers notifications to your backend.
1

Open Topic Subscriptions

In the SNS console, click on your topic → Create subscription
2

Configure Subscription

  • Topic ARN: Select your topic (e.g., owear)
  • Protocol: HTTPS
  • Endpoint: Your backend’s SNS notification URL, e.g.:
https://your-backend-domain.com/api/v1/sns/notification
3

Create Subscription

Click Create subscription. AWS will send a SubscriptionConfirmation request to your endpoint. The backend handles this automatically and confirms the subscription.
4

Verify Confirmation

After a few seconds, refresh the subscriptions page. The status should change from Pending confirmation to Confirmed.
The backend must be publicly accessible for SNS to reach it. If developing locally, use a tool like ngrok to expose your local server.

​
Step 6: Update Environment Variables

Add the AWS configuration to your .env file: 
# AWS S3 Configuration
AWS_BUCKET_NAME=open-wearables-xml
AWS_REGION=eu-north-1
AWS_ACCESS_KEY_ID=your-access-key-id
AWS_SECRET_ACCESS_KEY=your-secret-access-key

# SNS Configuration
AWS_SNS_TOPIC_ARN=arn:aws:sns:eu-north-1:123456789012:owear

​
Create IAM User with S3 and SNS Permissions

  1. Go to IAM Console
  2. Click Users → Add users
  3. User name: open-wearables-app
  4. Select Access key - Programmatic access
  5. Click Next: Permissions
  6. Attach policies:
    • AmazonS3FullAccess (or create a custom policy with specific bucket access)
    • AmazonSNSFullAccess (or create a custom policy with specific topic access)
  7. Click through to create the user
  8. Copy the Access key ID and Secret access key immediately (you won’t see the secret again)
Keep your AWS credentials secure! Never commit them to version control.

​
Step 7: Restart Services

After updating the .env file, restart your services: 
# Using Docker Compose
docker compose down
docker compose up -d

​
Verify Setup

Test your configuration:
1

Check SNS Subscription

In the SNS console, verify your subscription status is Confirmed.
2

Test Upload

Try uploading a small XML file using the presigned URL endpoint. The frontend handles this automatically, or you can use the upload script for manual testing:
python upload_xml.py test-export.xml
See the Apple XML Import Guide for details.
3

Check Backend Logs

After uploading, check the backend logs for SNS notification processing:
docker compose logs -f app
You should see logs about dispatching process_aws_upload tasks.

​
Troubleshooting

Possible causes:
  • Backend is not publicly accessible
  • Endpoint URL is incorrect
  • Backend returned an error during confirmation
Fix:
  • Verify your backend is reachable from the internet
  • Check the endpoint URL matches /api/v1/sns/notification
  • Check backend logs for confirmation errors
  • Delete the subscription and create a new one
Possible causes:
  • S3 event notification not configured correctly
  • SNS topic access policy doesn’t allow S3 to publish
  • Wrong bucket or topic ARN in configuration
Fix:
  • Verify the event notification in S3 bucket properties
  • Check the SNS access policy allows s3.amazonaws.com to publish
  • Test by manually uploading a file to S3 and checking backend logs
Possible causes:
  • IAM user doesn’t have required permissions
  • Wrong AWS credentials in .env
  • Bucket or topic in different region
Fix:
  • Verify IAM user has S3 and SNS permissions
  • Double-check AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  • Ensure AWS_REGION matches where your bucket and topic are located

​
Cost Optimization

S3 Storage

  • Uploaded XML files remain in S3 for your records
  • Set up Lifecycle Rules to archive or delete old files if needed
  • Consider moving to Glacier for long-term archival

SNS Notifications

  • SNS free tier includes 100,000 HTTP/HTTPS notifications per month
  • Check AWS SNS pricing for current limits and rates beyond free tier

Data Transfer

  • Keep S3 bucket and servers in the same AWS region
  • Use presigned URLs to avoid routing data through your server

Request Costs

  • S3 and SNS costs are negligible for typical usage
  • See AWS S3 pricing for current rates

​
Security Best Practices

  1. Use IAM Roles: If running on EC2/ECS, use IAM roles instead of access keys
  2. Restrict Bucket Access: Only allow specific IAM users/roles to access the bucket
  3. Enable Encryption: Use SSE-S3 or SSE-KMS for data at rest
  4. Monitor Access: Enable CloudTrail logging for S3 and SNS
  5. Rotate Credentials: Regularly rotate AWS access keys