|
1 | 1 | # lambda-python-starter |
2 | 2 |
|
3 | | -A clean starting point for Python AWS Lambda functions: structure, typed handlers, tests, local invoke and CI. |
| 3 | +A starting point for Python AWS Lambda services: two example handlers that share code, typed event models, JSON logging, validated configuration, tests, a local invoke script and a SAM template. |
4 | 4 |
|
5 | | -> **Project status: planning. There is no working code in this repository yet.** |
6 | | -> This README describes what the project will do. Nothing below can be installed or run today. |
7 | | -> Features are ticked off in the checklist as they are built. |
| 5 | +**Status:** everything in the checklist below works locally today. The tests and the local invoke run against [moto](https://github.com/getmoto/moto), an in-memory fake of AWS. **The SAM template has not been deployed.** It passes `cfn-lint` and is checked against the code by the tests; `sam validate`, `sam build` and `sam deploy` have not been run on it. |
8 | 6 |
|
9 | | -## What it is |
| 7 | +## Get started |
10 | 8 |
|
11 | | -A project template. |
| 9 | +Create your own repository from this one ("Use this template" on GitHub, or clone it), then: |
12 | 10 |
|
13 | | -**The problem it solves:** Every new Lambda project repeats the same setup for packaging, testing and local runs. |
| 11 | +```bash |
| 12 | +python -m venv .venv && . .venv/bin/activate |
| 13 | +make install |
| 14 | +make test && make invoke EVENT=events/api.json |
| 15 | +``` |
| 16 | + |
| 17 | +`make install` runs `pip install -e ".[dev]"`. To install without cloning: `pip install git+https://github.com/narayana006/lambda-python-starter`. |
| 18 | + |
| 19 | +`make invoke` runs a handler in your terminal with an event file. No AWS account, credentials or Docker are needed: DynamoDB is faked in memory and seeded from `events/seed_items.json`. Real output of `make invoke EVENT=events/api.json`: |
| 20 | + |
| 21 | +``` |
| 22 | +python scripts/invoke.py events/api.json |
| 23 | +--- invoking handlers.api.handler with events/api.json (in-memory AWS, logs follow) |
| 24 | +{"timestamp": "2026-11-10T16:35:14.859+00:00", "level": "INFO", "message": "request handled", "service": "lambda-python-starter", "logger": "handlers.api", "aws_request_id": "ad1a37ac-df91-4640-8eeb-5f2e2381e120", "function_name": "local-invoke", "cold_start": true, "route": "GET /items/{id}", "status": 200} |
| 25 | +--- result |
| 26 | +{ |
| 27 | + "statusCode": 200, |
| 28 | + "headers": { |
| 29 | + "content-type": "application/json" |
| 30 | + }, |
| 31 | + "body": { |
| 32 | + "id": "item-001", |
| 33 | + "name": "Blue widget", |
| 34 | + "quantity": 3 |
| 35 | + } |
| 36 | +} |
| 37 | +--- items in the in-memory table afterwards: 2 |
| 38 | +``` |
| 39 | + |
| 40 | +The log line is what the function would write to CloudWatch Logs. The timestamp and request id differ on each run. |
| 41 | + |
| 42 | +The handler is picked from the event shape, so an SQS event goes to the queue handler. Output of `make invoke EVENT=events/sqs.json` (the second of the three messages has an empty name on purpose): |
14 | 43 |
|
15 | | -## Who it is for |
| 44 | +``` |
| 45 | +python scripts/invoke.py events/sqs.json |
| 46 | +--- invoking handlers.queue.handler with events/sqs.json (in-memory AWS, logs follow) |
| 47 | +{"timestamp": "2026-11-10T16:35:15.532+00:00", "level": "WARNING", "message": "invalid message", "service": "lambda-python-starter", "logger": "handlers.queue", "aws_request_id": "aeab1a0d-43fe-4acc-a202-9b3eeed0f0bf", "function_name": "local-invoke", "cold_start": true, "message_id": "22222222-2222-4222-8222-222222222222", "reason": "'name' must be a non-empty string", "receive_count": 2} |
| 48 | +{"timestamp": "2026-11-10T16:35:15.533+00:00", "level": "INFO", "message": "batch processed", "service": "lambda-python-starter", "logger": "handlers.queue", "aws_request_id": "aeab1a0d-43fe-4acc-a202-9b3eeed0f0bf", "function_name": "local-invoke", "cold_start": true, "received": 3, "failed": 1} |
| 49 | +--- result |
| 50 | +{ |
| 51 | + "batchItemFailures": [ |
| 52 | + { |
| 53 | + "itemIdentifier": "22222222-2222-4222-8222-222222222222" |
| 54 | + } |
| 55 | + ] |
| 56 | +} |
| 57 | +--- items in the in-memory table afterwards: 4 |
| 58 | +``` |
| 59 | + |
| 60 | +Use `make invoke EVENT=events/sqs.json HANDLER=handlers.queue.handler` to choose the handler yourself. |
| 61 | + |
| 62 | +## Layout |
| 63 | + |
| 64 | +``` |
| 65 | +src/ |
| 66 | + handlers/ |
| 67 | + api.py HTTP API: POST /items, GET /items/{id} |
| 68 | + queue.py SQS consumer with partial batch failures |
| 69 | + shared/ |
| 70 | + config.py settings from environment variables, validated once |
| 71 | + log.py JSON log formatter and per-invocation context |
| 72 | + events.py HttpRequest, SqsRecord, LambdaContext, json_response |
| 73 | + middleware.py @lambda_handler: config, logging context, error logging |
| 74 | + items.py the example model and its DynamoDB storage |
| 75 | + requirements.txt runtime dependencies for `sam build` (empty: boto3 is in the runtime) |
| 76 | +events/ sample events and seed data |
| 77 | +scripts/invoke.py local invoke |
| 78 | +tests/ pytest suite |
| 79 | +template.yaml SAM template |
| 80 | +Makefile |
| 81 | +``` |
| 82 | + |
| 83 | +`src/` is what gets packaged (`CodeUri: src/`), so handler paths are `handlers.api.handler` and `handlers.queue.handler`. |
16 | 84 |
|
17 | | -Developers starting a new Python Lambda service. |
| 85 | +## What the pieces do |
18 | 86 |
|
19 | | -## How it will work |
| 87 | +**Handlers.** `api.py` maps route keys to functions. A bad request becomes a 400 with a message, an unknown route a 404, and an unexpected error a 500 that carries only the request id while the traceback goes to the log. `queue.py` stores each message and returns `batchItemFailures`, so one bad message does not make the whole batch retry. |
20 | 88 |
|
21 | | -1. You create a repository from the template. |
22 | | -2. You replace the example handler with your own. |
23 | | -3. You test locally and deploy with the included template. |
| 89 | +**Typed events.** `HttpRequest.from_event` reads an API Gateway HTTP API event (payload format 2.0): method, path, route key, path and query parameters, lower-cased headers, and the body, base64-decoded when needed. `SqsRecord` exposes the message id, body and receive count. Only the fields the handlers use are modelled. |
24 | 90 |
|
25 | | -## Planned features |
| 91 | +**Configuration.** `Config.from_env` reads these variables and fails on the first invocation with a clear message if one is wrong: |
26 | 92 |
|
27 | | -Unticked items are not built yet. |
| 93 | +| Variable | Required | Default | Meaning | |
| 94 | +| --- | --- | --- | --- | |
| 95 | +| `TABLE_NAME` | yes | | DynamoDB table for items | |
| 96 | +| `LOG_LEVEL` | no | `INFO` | `DEBUG`, `INFO`, `WARNING`, `ERROR` or `CRITICAL` | |
| 97 | +| `SERVICE_NAME` | no | `lambda-python-starter` | value of the `service` log field | |
28 | 98 |
|
29 | | -- [ ] Project layout with shared code and several handlers |
30 | | -- [ ] Typed event models |
31 | | -- [ ] Unit tests with sample events |
32 | | -- [ ] Local invoke script |
33 | | -- [ ] Structured JSON logging |
34 | | -- [ ] CI workflow and a SAM template |
| 99 | +**Logging.** One JSON object per line with `timestamp`, `level`, `message`, `service`, `logger`, `aws_request_id`, `function_name` and `cold_start`. Anything passed as `extra={...}` becomes a top-level field: |
| 100 | + |
| 101 | +```python |
| 102 | +logger.info("item created", extra={"item_id": item.id}) |
| 103 | +``` |
35 | 104 |
|
36 | | -## Planned usage |
| 105 | +Exceptions are logged with the traceback in an `exception` field. Request bodies are not logged. `boto3`, `botocore` and `urllib3` loggers are kept at `WARNING` or above. |
37 | 106 |
|
38 | | -This is the intended interface, shown so you can judge the design. **It does not work yet**, and names may change before the first release. |
| 107 | +**`@lambda_handler`.** Wraps each handler: loads the configuration, sets up logging, attaches the request context to every log line, and logs then re-raises unhandled exceptions so Lambda still records the invocation as failed. |
| 108 | + |
| 109 | +## Tests |
39 | 110 |
|
40 | 111 | ```bash |
41 | | -make test && make invoke EVENT=events/api.json |
| 112 | +make test |
42 | 113 | ``` |
43 | 114 |
|
44 | | -## Contributing |
| 115 | +69 tests. `tests/conftest.py` sets dummy credentials for every test and points the AWS config files at empty paths. Handlers are called with the files in `events/` and a moto DynamoDB table. `tests/test_project.py` checks that the template and the code agree: handler paths resolve to real functions, the routes in the template match the routes in `api.py`, the environment variables match what `config.py` reads, and the IAM statements are limited to the table. |
| 116 | + |
| 117 | +## The SAM template |
| 118 | + |
| 119 | +`template.yaml` defines: |
| 120 | + |
| 121 | +- `ApiFunction` behind an HTTP API with `POST /items` and `GET /items/{id}` |
| 122 | +- `QueueFunction` reading `ItemsQueue` in batches of 10 with `ReportBatchItemFailures` |
| 123 | +- `ItemsTable`, a DynamoDB table with on-demand billing |
| 124 | +- `ItemsDeadLetterQueue`, which receives a message after three failed deliveries |
| 125 | + |
| 126 | +Permissions the functions get: |
| 127 | + |
| 128 | +| Function | IAM actions | Resource | |
| 129 | +| --- | --- | --- | |
| 130 | +| `ApiFunction` | `dynamodb:GetItem`, `dynamodb:PutItem` | `ItemsTable` | |
| 131 | +| `QueueFunction` | `dynamodb:PutItem` | `ItemsTable` | |
| 132 | + |
| 133 | +SAM also adds the basic execution role for CloudWatch Logs to both functions and the SQS polling permissions to `QueueFunction`. |
| 134 | + |
| 135 | +How far the template has been checked: |
45 | 136 |
|
46 | | -Feedback on the design is the most useful contribution right now. Open an issue to suggest a feature, question an approach or describe your use case. Code contributions are welcome once the first version lands. |
| 137 | +- `make lint-template` (`cfn-lint template.yaml`, after `pip install cfn-lint`) passes with cfn-lint 1.60.0. It runs offline. |
| 138 | +- The tests described above pass. |
| 139 | +- `make validate`, `make build` and `make deploy` call the SAM CLI. They are provided but **have not been run**, and the stack has never been deployed. Expect to fix something on your first deploy. |
| 140 | + |
| 141 | +`make deploy` creates billable resources in your account. The HTTP API has no authorizer, so anyone who knows the URL can call it; add one before putting real data behind it. |
| 142 | + |
| 143 | +## Making it yours |
| 144 | + |
| 145 | +1. Replace `src/shared/items.py` with your own model and storage. |
| 146 | +2. Edit the routes in `src/handlers/api.py`, or add a module under `src/handlers/` and a matching function in `template.yaml`. |
| 147 | +3. Add an event file under `events/` for each new case and use it in a test and with `make invoke`. |
| 148 | +4. List runtime dependencies in `src/requirements.txt` and in `pyproject.toml`. |
| 149 | + |
| 150 | +## Checklist |
| 151 | + |
| 152 | +- [x] Project layout with shared code and several handlers |
| 153 | +- [x] Typed event models (HTTP API v2 and SQS) |
| 154 | +- [x] Unit tests with sample events |
| 155 | +- [x] Local invoke script |
| 156 | +- [x] Structured JSON logging |
| 157 | +- [x] Validated configuration from environment variables |
| 158 | +- [x] CI workflow and a SAM template (template linted, not deployed) |
| 159 | +- [ ] Event models for more sources (EventBridge, S3, DynamoDB streams) |
| 160 | +- [ ] Tracing and metrics |
| 161 | +- [ ] Deployment workflow |
| 162 | + |
| 163 | +## Limitations |
| 164 | + |
| 165 | +- Never deployed. Behaviour inside the real Lambda runtime (the log handler replacement, cold start flag, IAM permissions, API Gateway integration) is untested there. |
| 166 | +- Local invoke always uses the in-memory fake. It cannot call real AWS, and it does not emulate API Gateway routing, timeouts or memory limits. Use `sam local invoke` for a closer emulation. |
| 167 | +- The API handler has no authentication and no pagination or list endpoint. |
| 168 | +- `ItemStore.put` replaces an existing item with the same id. |
| 169 | +- The CI workflow runs the tests only. It has not run on GitHub yet. |
| 170 | + |
| 171 | +## Contributing |
47 | 172 |
|
48 | | -## Author |
| 173 | +Issues and pull requests are welcome. Keep the starter small: a change should be something most new Lambda projects need, with a test and a sample event. |
49 | 174 |
|
50 | 175 | Built and maintained by [Narayana Vemburaj](https://github.com/narayana006). |
51 | 176 |
|
|
0 commit comments