# AWS Emulator vs LocalStack and moto: catches IAM denies, misses a boundary

> In our run, AWS Emulator 1.0.505 returned 403 for an identity-policy Deny and a bucket-policy Deny that LocalStack 4.14.0 allowed on default settings. All three tools, moto included, let a permissions boundary through.

**Author:** LAXIMA Team  
**Published:** 2026-11-18  
**Updated:** 2026-11-18  
**Reading time:** 22 min  
**Category:** technology  
**Tags:** AWS Emulator, LocalStack, moto, IAM, local AWS testing, AI agents, developer tools  
**Canonical URL:** https://laxima.tech/blog/aws-emulator-vs-localstack-moto-tested

---
In our side-by-side run of [AWS Emulator](https://github.com/aws/aws-emulator) vs LocalStack and moto, the emulator (1.0.505) returned `403 AccessDenied` for an identity-policy Deny and an S3 bucket-policy Deny that LocalStack 4.14.0 let through on default settings. The identity Deny only applies when a test signs as an IAM user created inside the emulator. All three tools let through a `GetObject` that a permissions boundary excludes.

Use it for deny tests written that way. We probed S3, and AWS's own v1.0.505 scores pass its DynamoDB and SQS resource-policy deny tests. Don't use it to prove least privilege. Three more places a green run misleads: its `python3.12` Lambda ran on RustPython `3.14.0.alpha`, `CreateQueue` returns a real `sqs.us-east-1.amazonaws.com` URL that some SDKs follow to real AWS, and [one tester](https://github.com/aws/aws-emulator/issues/503) found 319 of 363 CloudFormation resource types without a provisioner reach `CREATE_COMPLETE` with nothing created.

AWS [made it generally available](https://aws.amazon.com/blogs/aws/test-your-applications-against-aws-locally-with-aws-emulator/) on 11 November with no service fee. We built its image from the documented Dockerfile and ran it beside LocalStack 4.14.0, the last community tag that starts without a token, and a moto server. One 4 vCPU Linux VM, one 28-call boto3 workload plus IAM and Lambda probes, every tool on its defaults. This is a comparison of defaults. The emulator enforces IAM with no flag. moto has a free opt-in check (`INITIAL_NO_AUTH_ACTION_COUNT`) that we left off, and LocalStack's `ENFORCE_IAM` [needs a paid plan](https://docs.localstack.cloud/aws/developer-tools/security-testing/iam-policy-enforcement/). We never called real AWS: every "AWS denies this" below is AWS's documented policy logic.

## The emulator enforces explicit denies with no flag

We gave one IAM user `Allow s3:*` and `Deny s3:GetObject`. On the emulator its `GetObject` got an AWS-style "is not authorized to perform: s3:GetObject" message naming the explicit deny, and `SimulatePrincipalPolicy` returned `explicitDeny`. LocalStack and moto returned the object.

<table class="blog-table" style="min-width: 100px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th class="blog-table-header" colspan="1" rowspan="1"><p>Probe (expected AWS result)</p></th><th class="blog-table-header" colspan="1" rowspan="1"><p>AWS Emulator 1.0.505 (default)</p></th><th class="blog-table-header" colspan="1" rowspan="1"><p>LocalStack 4.14.0 (default, <code>ENFORCE_IAM</code> off)</p></th><th class="blog-table-header" colspan="1" rowspan="1"><p>moto server (default, auth off)</p></th></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Identity-policy explicit Deny on <code>GetObject</code> (AccessDenied)</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>403 AccessDenied</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>200, allowed</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>200, allowed</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p><code>SimulatePrincipalPolicy</code></p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><code>explicitDeny</code></p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>500 InternalError</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>HTML 500, unparseable</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Bucket-policy Deny, IAM user and the tool's default key (AccessDenied)</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Denied for both</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Allowed for both</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Denied, code <code>403</code> not <code>AccessDenied</code></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Permissions boundary allowing only <code>s3:ListBucket</code>, then <code>GetObject</code> (AccessDenied)</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><strong>Allowed</strong></p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Allowed</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Allowed</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Lambda role with no policy reads S3, then after a grant (denied, then allowed)</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Denied, then allowed</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Allowed both times</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Not measured</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>DynamoDB <code>CreateTable</code> named <code>ab</code> (ValidationException)</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>AWS's exact message</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Right code, different wording</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>Accepted, table created <code>ACTIVE</code></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Missing key, failed conditional put, missing queue, missing function</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>All match</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>All match</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>All match</p></td></tr></tbody></table>

The Lambda probe tested a missing permission, not an explicit Deny. AWS's own test of an explicit Deny on a function's role, `iam__lambda__sts.function-identity-explicit-deny-overrides-its-allow`, still fails in the [v1.0.505 scores](https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz). It is one of 5 failures among the 30 tests with "explicit-deny" in their ID. We have no moto reading for in-function IAM: our function got `http://host.docker.internal:5000`, which doesn't resolve on Linux's default Docker bridge.

One condition decides whether your suite gets any of this. The emulator's default key speaks for the account, the position AWS root credentials are in. Per the [S3 authorization code](https://github.com/aws/aws-emulator/blob/eb0f368660289f69b8f99c1e01829dfc9eac6bcd/services/service-com-amazonaws-s3-amazons3/src/service/request_authorization.rs), such a request "is allowed where no readable document denies it". Identity policies apply only when a test signs as a user or role created inside the emulator, as our probes did. A suite that runs everything on the default credentials never exercises identity policies. At most it catches resource-policy denies, like the bucket policy in our run.

Two builders saw the same ordering before us. [DevOps Daily](https://devops-daily.com/posts/aws-emulator-hands-on-test) ran 32 permission cases on v1.0.322, with expected results taken from AWS docs rather than real AWS: the emulator matched 27, moto 8 and Floci 6. Jared Lynskey's real bucket policy, which would have broken on S3, got a 403 from the emulator alone out of four tools: "[every one of them would have let me ship the broken policy.](https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/)"

The bucket-policy result contradicts AWS's own [security page](https://docs.aws.amazon.com/emulator/latest/userguide/security.html): "The emulator never denies a request signed with its own key. It doesn't check AWS KMS key policies or Amazon S3 bucket policies." The S3 source for the build we ran disagrees: "[An explicit deny wins over every grant, including the account's own credential](https://github.com/aws/aws-emulator/blob/eb0f368660289f69b8f99c1e01829dfc9eac6bcd/services/service-com-amazonaws-s3-amazons3/src/service/request_authorization.rs)". A comparison article that trusted the docs warns of a [false pass on bucket reads](https://andrew.ooo/answers/aws-emulator-vs-localstack-vs-moto-local-aws-testing-2026/). For explicit bucket-policy denies on 1.0.505, that warning is out of date. Either the page or the code will change; check the version you pin.

## A permissions boundary stopped nothing on any of the three

A user with `Allow s3:*` and a boundary that allows only `s3:ListBucket` could still `GetObject` on all three tools. The boundary was a customer-managed policy we created and passed at `CreateUser`. Our probe, trimmed:

```python
import json, boto3
iam, s3 = boto3.client("iam"), boto3.client("s3")
doc = lambda action: json.dumps({"Version": "2012-10-17", "Statement": [
    {"Effect": "Allow", "Action": action, "Resource": "*"}]})
s3.create_bucket(Bucket="pb-test"); s3.put_object(Bucket="pb-test", Key="k", Body=b"v")
pb = iam.create_policy(PolicyName="boundary", PolicyDocument=doc("s3:ListBucket"))["Policy"]["Arn"]
iam.create_user(UserName="bounded", PermissionsBoundary=pb)
iam.put_user_policy(UserName="bounded", PolicyName="p", PolicyDocument=doc("s3:*"))
k = iam.create_access_key(UserName="bounded")["AccessKey"]
u = boto3.client("s3", aws_access_key_id=k["AccessKeyId"], aws_secret_access_key=k["SecretAccessKey"])
u.get_object(Bucket="pb-test", Key="k")  # AWS: AccessDenied. v1.0.505: returns the object
```

AWS's own `sts.permissions-boundary-denies-an-action-the-roles-own-identity-policy-allows` also still fails in the [v1.0.505 scores](https://github.com/aws/aws-emulator/releases/tag/v1.0.505). Boundaries aren't ignored everywhere, though. In the same scores `iam.permissions-boundary-caps-a-real-signed-call` passes. That test differs from our probe in two ways: it calls IAM's own API rather than S3, and it attaches the boundary afterwards with `PutUserPermissionsBoundary`. We can't tell which difference matters. Probe the service and the attachment path your code actually uses. We found no open issue for the plain IAM-user case.

Session policies are half-fixed, and the state moves by the hour. An open CRITICAL issue lists [nine root causes](https://github.com/aws/aws-emulator/blob/eb0f368660289f69b8f99c1e01829dfc9eac6bcd/issues/things-that-do-not-work/20261029T192356Z-iam-9-net-new-authorization-and-policy-evaluation.md) of authorization gaps: AssumeRole's inline session policy is ignored, and IAM's own API performs no authorization for assumed-role sessions (`ListAttachedRolePolicies` returns 200 "for a session AWS refuses"). Between v1.0.498 and v1.0.505, about 10 hours apart, the `iam:PolicyARN` condition-key test and three STS session-policy tests started passing, and one other session-policy test started failing. The two that matter most for least privilege still fail: `iam.session-policy-intersects-the-role-policy` and `iam.assumed-role-session-is-authorized-by-the-role-policy`.

Practitioners found two more routes around a deny. In DevOps Daily's run, a user with no policy assumed an account-trusted admin role and created an IAM user. In [#380](https://github.com/aws/aws-emulator/issues/380), CloudFormation `CreateStack` without a `RoleARN` created a bucket for a user whose identity policy denies `s3:CreateBucket`. And from the source, which we read but did not run: the emulator holds [138 AWS-managed policies](https://github.com/aws/aws-emulator/blob/eb0f368660289f69b8f99c1e01829dfc9eac6bcd/services/service-com-amazonaws-iam-awsidentitymanagementv20100508/src/state/authorization.rs) out of AWS's thousand-plus, and fails open for a principal holding one it lacks, such as `AmazonDynamoDBFullAccess`.

## A local AccessDenied can be wrong too

Five separate GitHub authors ([#369](https://github.com/aws/aws-emulator/issues/369), [#459](https://github.com/aws/aws-emulator/issues/459), [#451](https://github.com/aws/aws-emulator/issues/451), [#447](https://github.com/aws/aws-emulator/issues/447), [#409](https://github.com/aws/aws-emulator/issues/409)) report that the standard `aws:SecureTransport` Deny blocks HTTPS calls it should allow. The calls hit include S3-to-SQS notifications, DynamoDB `PutItem`, KMS `CreateKey`, CloudTrail `CreateTrail` and Lambda SQS event source mappings. The author of #369 ran the same commands in real us-east-1, where they succeed. #451 traces it to the request context never supplying `aws:SecureTransport`. The condition stays undecided and is read as a refusal.

If your bucket and queue policies carry that statement, as CDK's `enforceSSL` option adds, expect false reds until it's fixed. DevOps Daily's rule covers both directions: "[Treat a local](https://devops-daily.com/posts/aws-emulator-hands-on-test) `AccessDenied` [as a bug, and a local success as unproven.](https://devops-daily.com/posts/aws-emulator-hands-on-test)"

## Your python3.12 Lambda runs on RustPython 3.14 alpha

Our probe function, created with `Runtime=python3.12`, reported `sys.implementation.name = 'rustpython'` and `sys.version = '3.14.0.alpha'`. Its `boto3` is a shim with no `__version__`, and code that reads `boto3.__version__` raises `AttributeError`; our own first probe crashed on it. There is no `AWS_ACCESS_KEY_ID` in the function's environment. `import sqlite3` and `import zoneinfo` both fail.

LocalStack and moto ran real CPython 3.12.16 with boto3 1.42.97, in separate runtime images. According to the [Lambda service code](https://github.com/aws/aws-emulator/blob/eb0f368660289f69b8f99c1e01829dfc9eac6bcd/services/service-com-amazonaws-lambda-awsgirapiservice/src/state/execution.rs), Python runs on RustPython and Node.js on QuickJS inside the emulator process. Only Zip packages for those two families run. Container images, Java, Go, .NET and Ruby get `InvalidRuntimeException`. Neither the launch post nor the docs mention the interpreter.

Builders ran into it anyway. yythzy had to [bundle signature verification](https://zenn.dev/yythzy/articles/aws-emulator-02) into a Node function, and [#28](https://github.com/aws/aws-emulator/issues/28) lists missing Node built-ins such as `node:http` and `fetch`. [#332](https://github.com/aws/aws-emulator/issues/332), reported on v1.0.338 and still open, shows memory growing about 7 MiB per Python invoke and never coming back.

The trade is footprint. There is no runtime image to pull, and a cold echo invoke took 131 to 158 ms against 447 to 491 ms on LocalStack.

## CreateQueue hands your SDK a real AWS address

`CreateQueue` on the emulator returned `https://sqs.us-east-1.amazonaws.com/000000000000/<name>`, the real AWS format. LocalStack and moto return local hosts. With boto3 1.44.5 and `AWS_ENDPOINT_URL` set, our send and receive still went to the emulator and got the message back.

Other SDKs follow the host. DevOps Daily found that the [JS SDK v3 SQS client](https://devops-daily.com/posts/aws-emulator-hands-on-test) sends to real SQS by default, and [#25](https://github.com/aws/aws-emulator/issues/25) reports the same for Java SDK 1.x and JS v2. With real keys in the CI environment, those calls would be signed and sent to your account. In the JS SDK v3, set `useQueueUrlAsEndpoint: false`. With any SDK, [keep real credentials out of the test job](https://laxima.tech/blog/ai-agent-security-after-hugging-face-intrusion).

## Half a second to start, nearly double LocalStack's idle RAM

Three rounds per tool on the same VM, a fresh container each time, Lambda runtime images already pulled. The "first run" row is one run with each tool's own image present and no Lambda runtime image cached.

<table class="blog-table" style="min-width: 100px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th class="blog-table-header" colspan="1" rowspan="1"><p>Measurement</p></th><th class="blog-table-header" colspan="1" rowspan="1"><p>AWS Emulator 1.0.505</p></th><th class="blog-table-header" colspan="1" rowspan="1"><p>LocalStack 4.14.0</p></th><th class="blog-table-header" colspan="1" rowspan="1"><p>moto server</p></th></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p><code>docker run</code> to healthy</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>0.51 to 0.57 s</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>2.91 to 3.22 s</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>1.22 to 1.31 s</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Idle RAM</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>185 MiB</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>96 MiB</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>45 MiB</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>RAM after the 28-call workload</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>211 MiB</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>371 MiB + 15 MiB Lambda container</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>89 MiB</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Workload wall time</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>4.32 to 4.34 s</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>6.05 to 6.30 s</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>3.03 to 3.26 s</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>First run: tool image present, Lambda runtime image not cached*</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>4.35 s</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>19.0 s</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>25.2 s</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Lambda cold invoke (echo)</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>131 to 158 ms</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>447 to 491 ms</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>890 to 949 ms</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Image on disk</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>1.98 GB, self-built</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>1.77 GB</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>557 MB</p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Extra Lambda image</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>none</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>754 MB</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p>1.2 GB</p></td></tr></tbody></table>

\*Obtaining each tool's own image is not included: our emulator image build took 1m21s, and we didn't time the pulls of LocalStack's or moto's images. LocalStack spent 13.8 s of its run creating the function while it pulled a 754 MB Lambda image, and moto 23.0 s on its first invoke while it pulled 1.2 GB.

The emulator's first-run lead comes from needing no Lambda runtime image. On a runner with nothing cached, add the image build, or the package install below, before you compare. With everything warm, the emulator loses to moto on wall time because it copies AWS: three of its 4.3 seconds are the DynamoDB table sitting in `CREATING`, and one is Lambda in `Pending`. moto reports `ACTIVE` at once, which hides waiter bugs. Jared Lynskey measured the same RAM ordering on Mac Docker: [176 MiB idle for the emulator, 90 for LocalStack 4.12, 45 for moto](https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/).

The 2 GiB cap in AWS's [documented run command](https://docs.aws.amazon.com/emulator/latest/userguide/getting-started-docker.html) is a hard edge: past it the container exits with code 137 and every emulated resource is gone. The emulator also has limits AWS doesn't, among them [1,000 objects per bucket](https://github.com/aws/aws-emulator/issues/30), [64 MiB per bucket](https://github.com/aws/aws-emulator/blob/eb0f368660289f69b8f99c1e01829dfc9eac6bcd/services/service-com-amazonaws-s3-amazons3/src/state.rs) and 1,000 messages per queue. Memory isn't handed back either: after deleting everything, one tester's instance [still held 278.8 MiB](https://devops-daily.com/posts/aws-emulator-hands-on-test).

## Wiring it into CI takes more than an endpoint URL

There is [no prebuilt image](https://docs.aws.amazon.com/emulator/latest/userguide/getting-started-docker.html): you build one around a 1.1 GB binary. The usual `test`/`test` dummy keys get `403 AccessDenied`. LocalStack-style CI configs have to point at the per-run credentials file and CA bundle instead. Our wiring:

```bash
mkdir -p "$HOME/.aws/emulator"
docker run -d --memory=2g --memory-swap=2g -p 127.0.0.1:9297:9297 \
  --user "$(id -u):$(id -g)" -e HOME="$HOME" \
  -v "$HOME/.aws/emulator:$HOME/.aws/emulator" \
  aws-emulator -p 9297 --telemetry off

E="$HOME/.aws/emulator/9297"
until [ -f "$E/ca.pem" ] && curl -sf --cacert "$E/ca.pem" https://127.0.0.1:9297/__admin/health; do sleep 0.1; done
# test job only: left set, AWS_DISABLE_HOST_PREFIX_INJECTION breaks host-prefixed calls against real AWS
export AWS_ENDPOINT_URL=https://127.0.0.1:9297 \
  AWS_SHARED_CREDENTIALS_FILE="$E/emulator_credentials" \
  AWS_CONFIG_FILE="$E/config" AWS_CA_BUNDLE="$E/ca.pem" \
  AWS_DISABLE_HOST_PREFIX_INJECTION=true
```

Each start writes [new keys and a new CA](https://docs.aws.amazon.com/emulator/latest/userguide/cli-reference.html) to `~/.aws/emulator/9297/`. Recreate SDK clients after every restart, or the next call fails TLS and auth.

On Linux runners, skip the image build and install AWS's [RPM or deb](https://docs.aws.amazon.com/emulator/latest/userguide/setting-up.html). The x86\_64 RPM is 194 MB and downloaded in 7.9 s here, against 1m21s for our image build. Luka Kladarić's job runs [36 checks in under 3 minutes on a stock GitHub runner](https://chaosguru.substack.com/p/aws-finally-made-an-official-local) and caught all six bugs he planted. The TLS certificate covers only `localhost`, `127.0.0.1` and `::1` ([#23](https://github.com/aws/aws-emulator/issues/23)). A Compose or service-container setup that reaches the emulator as `emulator:9297` fails verification, which means the emulator has to run on the job host itself.

An unofficial multi-arch image [exists](https://github.com/aws/aws-emulator/issues/14) for macOS users who'd rather not build. AWS doesn't build or sign it; if you use it, pin it by digest.

## Check your own operations against the fidelity scores

AWS says every test in its public suite was [verified against at least two real AWS accounts](https://github.com/aws/aws-emulator-evaluation-suite). No earlier local emulator published anything like it. The [launch post](https://aws.amazon.com/blogs/aws/test-your-applications-against-aws-locally-with-aws-emulator/) promises a per-service score per release. What the [release page](https://github.com/aws/aws-emulator/releases/tag/v1.0.505) carries is a list of fix commits and a `scores.json.gz` of 46,026 per-test outcomes, with no per-service summary. Our own aggregation of that file for the build we ran: 32,751 pass, 13,188 fail, 87 skipped (71.2%).

<table class="blog-table" style="min-width: 50px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th class="blog-table-header" colspan="1" rowspan="1"><p>Service</p></th><th class="blog-table-header" colspan="1" rowspan="1"><p>Pass rate on v1.0.505, tests naming only this service (our own count)</p></th></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>SNS</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">94.6% (280/296)</a></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>KMS</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">91.0% (305/335)</a></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>S3</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">89.4% (1,158/1,295)</a></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>SQS</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">89.1% (320/359)</a></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>IAM</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">87.3% (635/727)</a></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>DynamoDB</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">77.1% (1,620/2,100)</a></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>Lambda</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">61.6% (499/810)</a></p></td></tr><tr><td class="blog-table-cell" colspan="1" rowspan="1"><p>CloudFormation</p></td><td class="blog-table-cell" colspan="1" rowspan="1"><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-brand-red hover:underline" href="https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz">18.2% (480/2,639)</a></p></td></tr></tbody></table>

CloudFormation's low figure is mostly one family of checks. In our own count, 2,031 of its 2,639 units are `DescribeType` catalog checks, one per resource type, and every one fails. Without them, 480 of the remaining 608 pass (79%). AWS's public single-service CloudFormation tests pass 350/427 (82.0%) on the same release, a different method.

Quote any of these numbers with its release and counting method. Better, check the operations your own code calls. AWS says its `aws-emulator` [agent skill](https://docs.aws.amazon.com/emulator/latest/userguide/getting-started-agent.html) checks a project against the manifest and the scores; we didn't run it. By hand, join the suite's `manifest.jsonl` to the release's scores:

```python
import gzip, json
ops = {"sqs:SendMessage", "dynamodb:Query", "s3:GetObject"}  # what your code calls
res = {u["unitId"]: u["outcome"] for u in json.load(gzip.open("scores.json.gz"))["units"]}
for line in open("manifest.jsonl"):  # from aws-emulator-evaluation-suite
    t = json.loads(line)
    if ops & set(t["operations"]):
        print(res.get(t["check_id"], "unscored"), t["check_id"])
```

The script lists every test that touches one of your operations, and its totals won't match the table. For a census of 501 stubs, `grep -rl 'SCAFFOLDED, NOT IMPLEMENTED' services/<service-crate>` in the repo is the reliable route.

Our own counts from the scores files and the repo turned up three things the docs don't say:

-   **Most failures aren't 501s.** Only 15.9% of failing wire tests in the [v1.0.505 scores](https://github.com/aws/aws-emulator/releases/download/v1.0.505/scores.json.gz) return `501 NotImplemented`. For S3, DynamoDB, SQS and IAM none do: those failures are wrong bodies and statuses.
    
-   **Passing tests do regress.** 201 tests with the same ID went from pass to fail between [v1.0.322](https://github.com/aws/aws-emulator/releases/download/v1.0.322/scores.json.gz) and [v1.0.506](https://github.com/aws/aws-emulator/releases/download/v1.0.506/scores.json.gz), though the docs say "[Fidelity tests that pass don't regress.](https://docs.aws.amazon.com/emulator/latest/userguide/versioning.html)" Some may be rebuilt or flaky tests. The scores record no suite version, and nobody outside AWS can tell which.
    
-   **The known-issues folder is stale both ways.** At commit `eb0f368`, 197 of the 624 open stub issues in `issues/things-that-do-not-work` name operations already implemented, while 7,234 [handlers](https://github.com/aws/aws-emulator/tree/eb0f368660289f69b8f99c1e01829dfc9eac6bcd/services) still carry the 501 scaffold marker.
    

## 319 CloudFormation resource types report CREATE\_COMPLETE with nothing behind them

GitHub user sstosh [deployed all 363 catalogued resource types](https://github.com/aws/aws-emulator/issues/503) that have no provisioner, on v1.0.484/485. 319 reached `CREATE_COMPLETE` with nothing created in the owning service, and none failed. Earlier reports found the same with CloudFront ([#202](https://github.com/aws/aws-emulator/issues/202): five resources, none in CloudFront), [CloudWatch alarms, ELB and ECS](https://www.reddit.com/r/aws/comments/1xua13r/i_tested_aws_emulator_against_18_services_last/) and [Cognito](https://github.com/aws/aws-emulator/issues/259). One r/aws reply named the risk: "your tests go green on resources that were never created".

We didn't test Terraform, CDK or CloudFormation ourselves. The Terraform reports at least give you settings to start from:

-   **Security groups** fail for all six reporters on [#46](https://github.com/aws/aws-emulator/issues/46), still on v1.0.471. In one stack that blocked 26 of 88 resources.
    
-   **S3 buckets** fail on AWS provider 6.23.0 and later, because tag reads go through S3 Control ([#56](https://github.com/aws/aws-emulator/issues/56); 6.22.0 works). [tomoki33](https://zenn.dev/tomoki33/articles/aws-emulator-first-try) pins the provider to `~> 5.0`. [Nana\_777](https://qiita.com/Nana_777/items/3f3c71525d867b22c9f2) got 6.73.0 to apply with `skip_requesting_account_id` and `s3_use_path_style` set.
    
-   **DynamoDB tables** drift forever: `DescribeTable` returns `OnDemandThroughput` as `{-1, -1}` ([#37](https://github.com/aws/aws-emulator/issues/37), four reporters). tomoki33 adds `ignore_changes` for `on_demand_throughput`.
    

If an agent [writes your Terraform](https://laxima.tech/blog/building-the-ultimate-terraform-skill-for-claude-code-a-devops-guide) and checks its work against the emulator, it can get both kinds of wrong answer: a red security group on a valid config, and a green stack with nothing behind it. Check agent-written infrastructure code in a sandbox account.

## Only AWS's agents can change the code

AWS [says](https://www.aboutamazon.com/news/aws/how-autonomous-ai-agents-built-aws-emulator) "Our team hasn't written or reviewed any of the code". The [public history](https://github.com/aws/aws-emulator/commits/main) has more than 35,600 commits since 21 October, nearly all by IssueFinderAgent and IssueFixerAgent. That makes it [agent-written code whose only warranty is a published test score](https://laxima.tech/blog/ai-generated-code-is-cheap-reliability-isnt). AWS also says a merge gate rejects any change that breaks a passing fidelity test. Outsiders can't check that, and the 201 flips above don't settle it either way. The repo [takes no pull requests](https://github.com/aws/aws-emulator/blob/main/.github/CONTRIBUTING.md). A community root-cause fix for the Terraform security-group bug on #46 was declined, and the bug still reproduced on v1.0.471.

## LocalStack now sells what the emulator lacks

`localstack/localstack:latest` exited with code 55 in our run for lack of an auth token. The free comparison is the frozen 4.14.0 tag from February. LocalStack [ended Community support on 23 March 2026](https://blog.localstack.cloud/2026-upcoming-pricing-changes/), and Hobby is non-commercial. At [its list price](https://www.localstack.cloud/pricing), a 10-developer team on Base pays 10 × $39 × 12 = $4,680 a year by our own arithmetic, and Base is where IAM enforcement starts.

Its [docs](https://docs.localstack.cloud/aws/developer-tools/security-testing/iam-policy-enforcement/) say `ENFORCE_IAM` also evaluates permissions boundaries and SCPs, the control the emulator missed in our run. We didn't test it. LocalStack's [answer to the launch](https://blog.localstack.cloud/aws-emulator-is-here-what-you-need-to-know/) leans on that depth, "including areas like IAM", plus persistence. The pitch holds for now, and it rests on exactly the gaps listed above.

We didn't test Floci either, the free MIT-licensed alternative several HN commenters named. Two builders who did found it the lightest and fastest to start ([Jared Lynskey](https://jared.lynskey.co.nz/en/posts/2026/aws-emulator-setup/): 0.3 to 0.4 s, 11 MiB idle) and the weakest on permissions ([DevOps Daily](https://devops-daily.com/posts/aws-emulator-hands-on-test): 6 of 32 cases). One HN user [calls it unstable between releases](https://news.ycombinator.com/item?id=50457253).

AWS's evaluation runner takes any `--endpoint`. Rivals could be scored on the same [31,449 public tests](https://github.com/aws/aws-emulator-evaluation-suite); we haven't tried it.

## Replace LocalStack Community with it for S3, DynamoDB and SQS, not for least privilege

**On the frozen** `localstack:4.14.0` **tag because Hobby is non-commercial?** Move your S3, DynamoDB and SQS integration job now. The four common error cases we tried matched on all three tools. The emulator also returned AWS's exact DynamoDB validation text where moto created an invalid table, kept tables in `CREATING` long enough to catch waiter bugs, and passed 40 of 40 workflow checks in [DevOps Daily's run](https://devops-daily.com/posts/aws-emulator-hands-on-test) against moto's 32. Write deny tests to sign as users or roles created inside the emulator. We ran boto3 only, which kept SQS calls local; we didn't test the AWS CLI.

**On moto in-process (**`mock_aws`**) for fresh state per test?** Keep it for unit tests. The emulator has no reset route, as [jbmchuck pointed out](https://news.ycombinator.com/item?id=50457726). Outside its Rust-only embedded mode, a restart is the only way back to a clean state. Use the emulator for the integration job. On moto server, try its opt-in auth flag before you switch; we didn't test it.

**Need boundaries, session policies or SCPs checked?** No free tool we ran will do it. Use a sandbox account, or try LocalStack Base's `ENFORCE_IAM`.

Before you switch:

1.  **Pin the package version and its SHA-256**, and quote results with it. The Dockerfile pulls LTS/latest and [self-built images drift](https://techforward.io/aws-emulator-fidelity-scores-localstack-ci/); v1.0.505 and v1.0.506 shipped [99 minutes apart](https://github.com/aws/aws-emulator/releases) on 18 November. Pinning has a cost: [security fixes go only to the latest release](https://docs.aws.amazon.com/emulator/latest/userguide/versioning.html).
    
2.  **Keep real AWS keys out of the CI job**, and set `useQueueUrlAsEndpoint: false` in the JS SDK v3.
    
3.  **Turn telemetry off** with `--telemetry off` or `EMULATOR_TELEMETRY=off`. By default it reports CI usage and "[which AI coding assistant drives the emulator](https://docs.aws.amazon.com/emulator/latest/userguide/telemetry.html)".
    
4.  **Restart between suites** (0.55 s here) and rebuild SDK clients afterwards.
    
5.  **Budget memory.** If [#332](https://github.com/aws/aws-emulator/issues/332)'s leak of about 7 MiB per Python invoke still holds, a 2 GiB container starting at 211 MiB exits with code 137 after roughly (2,048 − 211) / 7 ≈ 260 Python invokes, by our own arithmetic. We didn't measure the leak. In a separate single run, our instance reached 378 MiB after four more Lambda probes and the IAM tests.
    

Don't trust a green run for:

-   permissions boundaries, session policies or AssumeRole scoping;
    
-   CloudFormation, CDK or SAM stacks;
    
-   Lambdas with native dependencies or `sqlite3`/`zoneinfo`, or runtimes other than Python and Node;
    
-   S3 paging past [1,000 objects](https://github.com/aws/aws-emulator/issues/30) or buckets over 64 MiB;
    
-   multi-account or cross-Region behaviour (one account, `000000000000`, per process);
    
-   Terraform configs with security groups.
    

For those, use a sandbox AWS account. One HN commenter [runs dedicated test accounts](https://news.ycombinator.com/item?id=50457958) for "$50 or so a month in fees". What would change our mind: the boundary probe above returning `AccessDenied`, `iam.session-policy-intersects-the-role-policy` and `iam.assumed-role-session-is-authorized-by-the-role-policy` passing in a release's scores, and Lambda running on real CPython or Node.

_Cover image: Screenshot: github.com/aws/aws-emulator, 18 November 2026._

### Is AWS Emulator a replacement for LocalStack?

For free commercial CI on S3, DynamoDB and SQS, largely yes. AWS Emulator is CC0-licensed with no service fee, while LocalStack's free Hobby tier is non-commercial. In our run it started in 0.55 s against LocalStack 4.14.0's 3 s, but idled at 185 MiB against 96 MiB. Don't rely on it for CloudFormation stacks or Lambda runtime behaviour.

### Does AWS Emulator enforce IAM policies?

For the explicit denies we tested, yes, when the test signs as an IAM user or role created inside AWS Emulator; its default key acts as account root. Version 1.0.505 enforced an identity-policy Deny and an S3 bucket-policy Deny, but let a permissions boundary through. AWS's own scores still fail 5 of its 30 explicit-deny tests.

### Does AWS Emulator's Lambda run real Python and Node.js?

No. AWS Emulator runs Python handlers on RustPython and Node.js handlers on QuickJS inside its own process. Our python3.12 function on version 1.0.505 reported Python 3.14.0 alpha and could not import sqlite3 or zoneinfo. Container-image functions and the Java, Go, .NET and Ruby runtimes are refused.

### Which AWS services does AWS Emulator support?

AWS Emulator publishes no service list. Each release ships a scores.json.gz of per-test results, covering 356 services for version 1.0.505. In our own count for that release, single-service tests pass at 94.6% for SNS, 89.4% for S3, 89.1% for SQS, 77.1% for DynamoDB, 61.6% for Lambda and 18.2% for CloudFormation.

### Can I use Terraform, CDK or CloudFormation with AWS Emulator?

You can deploy with them, but a successful deploy on AWS Emulator proves little. On versions 1.0.484 and 1.0.485, one GitHub reporter found 319 of 363 CloudFormation resource types without a provisioner reached CREATE\_COMPLETE with nothing created. Six reporters say Terraform security groups fail. We did not test infrastructure-as-code tools ourselves.

### How do I run AWS Emulator on macOS or Windows?

Build your own Docker image from the Dockerfile in AWS's user guide: AWS Emulator ships native packages only for Linux and no official prebuilt image. The documented run command caps the container at 2 GiB, and going over that limit exits with code 137 and loses all state. Sign requests with the per-run credentials file under ~/.aws/emulator/9297/.
