-
Notifications
You must be signed in to change notification settings - Fork 2
Sam/marketing api #77
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| # Edge Push Server Documentation Index | ||
|
|
||
| This file serves as the central index for all documentation in the Edge Push Server project. | ||
|
|
||
| ## Documentation Structure | ||
|
|
||
| ### Project Setup and Overview | ||
|
|
||
| #### `README.md` | ||
|
|
||
| - **When to read**: First time setting up the project, understanding basic server setup and deployment | ||
| - **Summary**: Project overview, setup instructions including AMQP/RabbitMQ 3.12 Docker configuration, PM2 management, and deployment procedures | ||
|
|
||
| ### Configuration Guides | ||
|
|
||
| #### `docs/guides/amqp-configuration.md` | ||
|
|
||
| - **When to read**: Before running `yarn start` for the first time, when setting up development environment, or troubleshooting message queue issues | ||
| - **Summary**: Complete guide for configuring the AMQP client connection required by the push server, including RabbitMQ 3.12 Docker setup, connection string format, security considerations, and troubleshooting tips | ||
|
|
||
| ### API Documentation | ||
|
|
||
| #### `docs/guides/marketing-api.md` | ||
|
|
||
| - **When to read**: When implementing marketing campaigns, sending location-based push notifications, or integrating with the marketing task queue system | ||
| - **Summary**: Complete documentation for the Marketing API endpoints including authentication, location targeting, task queue management, progress tracking, and integration examples. Covers device filtering, error handling, and best practices for large-scale push notification campaigns. | ||
|
|
||
| ### Technical References | ||
|
|
||
| #### `docs/guides/marketing-database-schema.md` | ||
|
|
||
| - **When to read**: When working on marketing system internals, database maintenance, or understanding the task queue architecture | ||
| - **Summary**: Technical documentation of database schema changes for the Marketing API, including ApiKey updates, MarketingTask structure, CouchDB views, migration considerations, and monitoring queries. Essential for developers working on the marketing system backend. | ||
|
|
||
| ### Migration Guides | ||
|
|
||
| #### `docs/guides/marketing-api-migration.md` | ||
|
|
||
| - **When to read**: When upgrading an existing Edge Push Server installation to include Marketing API support | ||
| - **Summary**: Step-by-step migration guide covering code updates, database migration, API key configuration, daemon setup, and verification procedures. Includes troubleshooting tips, rollback procedures, and post-migration tasks for existing installations. | ||
|
|
||
| ### Additional Resources | ||
|
|
||
| #### `docs/demo.ts` | ||
|
|
||
| - **When to read**: When learning how to integrate with the v2 API, testing push notifications | ||
| - **Summary**: Example TypeScript code demonstrating how to use the Edge Push Server v2 API | ||
|
|
||
| #### `docs/logrotate` | ||
|
|
||
| - **When to read**: When setting up production server, configuring log management | ||
| - **Summary**: Log rotation configuration for managing server logs in production | ||
|
|
||
| ## Quick Start | ||
|
|
||
| 1. Read `README.md` for project setup | ||
| 2. Configure AMQP by following `docs/guides/amqp-configuration.md` | ||
| 3. Set up your `pushServerConfig.json` with database and AMQP credentials | ||
| 4. Run `yarn install` and `yarn prepare` | ||
| 5. Start the server with `yarn start` | ||
|
|
||
| ## Architecture Overview | ||
|
|
||
| The Edge Push Server consists of: | ||
|
|
||
| - HTTP API server for device registration and notification triggers | ||
| - AMQP message queue for reliable message delivery | ||
| - Background daemons for processing notifications, price changes, and confirmations | ||
| - CouchDB for storing device registrations and settings | ||
| - Firebase Admin SDK for sending push notifications |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| # AMQP Client Configuration Guide | ||
|
|
||
| ## Overview | ||
|
|
||
| The Edge Push Server uses AMQP (Advanced Message Queuing Protocol) for reliable message delivery between the HTTP server and the push notification daemon. This guide explains how to configure the AMQP client before running the server. | ||
|
|
||
| ## What is AMQP Used For? | ||
|
|
||
| The AMQP client serves as the messaging backbone for the push notification system: | ||
|
|
||
| 1. **Message Queue**: The HTTP server enqueues push notification requests into a RabbitMQ/AMQP queue named "messages" | ||
| 2. **Decoupling**: Separates the HTTP API from the actual push notification delivery process | ||
| 3. **Reliability**: Ensures messages aren't lost if the push daemon is temporarily down | ||
| 4. **Load Management**: The queue prefetch is set to 50 messages to prevent overwhelming the push daemon | ||
|
|
||
| ## Architecture Flow | ||
|
|
||
| ``` | ||
| HTTP Server → AMQP Queue ("messages") → Publish Daemon → Firebase/Push Services | ||
| ``` | ||
|
|
||
| ## Configuration Requirements | ||
|
|
||
| ### 1. Create Configuration File | ||
|
|
||
| Before running `yarn start`, create a `pushServerConfig.json` file in the project root: | ||
|
|
||
| ```json | ||
| { | ||
| "listenHost": "127.0.0.1", | ||
| "listenPort": 8008, | ||
| "amqpUri": "amqp://username:password@localhost:5672", | ||
| "couchUri": "http://username:password@localhost:5984", | ||
| "currentCluster": "production" | ||
| } | ||
| ``` | ||
|
|
||
| ### 2. AMQP URI Format | ||
|
|
||
| The `amqpUri` follows the standard AMQP connection string format: | ||
|
|
||
| ``` | ||
| amqp://[username[:password]@]hostname[:port][/vhost] | ||
| ``` | ||
|
|
||
| Examples: | ||
|
|
||
| - Local development (Docker): `amqp://guest:guest@localhost:5672` | ||
| - Production with vhost: `amqp://edgeuser:securepass@rabbitmq.example.com:5672/edge` | ||
| - CloudAMQP: `amqp://user:pass@hostname.cloudamqp.com/instance` | ||
|
|
||
| ### 3. Required AMQP Server Setup | ||
|
|
||
| Before starting the Edge Push Server, ensure you have: | ||
|
|
||
| 1. **RabbitMQ Server version 3.12** (or compatible AMQP broker) running via Docker | ||
| 2. **User credentials** with permissions to: | ||
| - Create queues | ||
| - Publish messages | ||
| - Consume messages | ||
| - Set prefetch count | ||
| 3. **Network access** to the AMQP server on port 5672 | ||
|
|
||
| ### 4. Queue Configuration | ||
|
|
||
| The server automatically creates a queue named "messages" with: | ||
|
|
||
| - **Prefetch limit**: 50 messages (prevents memory overflow) | ||
| - **Manual acknowledgment**: Messages are only removed after successful processing | ||
| - **Durable**: Queue persists through server restarts (implementation dependent) | ||
|
|
||
| ## RabbitMQ Setup (Docker) | ||
|
|
||
| **Important**: The Edge Push Server requires RabbitMQ version 3.12 with management interface. | ||
|
|
||
| Run the following Docker command to set up RabbitMQ: | ||
|
|
||
| ```bash | ||
| docker run -d --name rabbitmq \ | ||
| -p 5672:5672 \ | ||
| -p 15672:15672 \ | ||
| -e RABBITMQ_DEFAULT_USER=guest \ | ||
| -e RABBITMQ_DEFAULT_PASS=guest \ | ||
| rabbitmq:3.12-management | ||
| ``` | ||
|
|
||
| This command: | ||
|
|
||
| - Uses RabbitMQ version 3.12 with management interface | ||
| - Exposes port 5672 for AMQP connections | ||
| - Exposes port 15672 for the web management UI | ||
| - Sets default credentials (guest/guest) | ||
|
|
||
| ### Cloud Services Alternative | ||
|
|
||
| - **CloudAMQP**: Managed RabbitMQ hosting | ||
| - **Amazon MQ**: AWS managed message broker | ||
| - **Azure Service Bus**: Alternative AMQP-compatible service | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Connection Errors | ||
|
|
||
| If you see connection errors when starting the server: | ||
|
|
||
| 1. **Check AMQP server is running**: | ||
|
|
||
| ```bash | ||
| # For Docker RabbitMQ | ||
| docker ps | grep rabbitmq | ||
| docker logs rabbitmq | ||
| ``` | ||
|
|
||
| 2. **Verify credentials**: | ||
|
|
||
| ```bash | ||
| # Test connection | ||
| curl -i -u username:password http://localhost:15672/api/overview | ||
| ``` | ||
|
|
||
| 3. **Check firewall/network**: | ||
| ```bash | ||
| telnet localhost 5672 | ||
| ``` | ||
|
|
||
| ### Common Issues | ||
|
|
||
| - **"Connection refused"**: AMQP server not running or wrong port | ||
| - **"Authentication failed"**: Incorrect username/password | ||
| - **"Access refused"**: User lacks necessary permissions | ||
| - **"Channel closed"**: Often indicates permission issues or resource limits | ||
|
|
||
| ## Security Considerations | ||
|
|
||
| 1. **Never commit** `pushServerConfig.json` with real credentials | ||
| 2. **Use strong passwords** for production AMQP instances | ||
| 3. **Enable TLS** for production: `amqps://` instead of `amqp://` | ||
| 4. **Limit network access** to AMQP ports using firewall rules | ||
| 5. **Use separate vhosts** for different environments (dev/staging/prod) | ||
|
|
||
| ## Monitoring | ||
|
|
||
| Monitor your AMQP queue health: | ||
|
|
||
| - **Queue depth**: Messages waiting to be processed | ||
| - **Consumer count**: Should match number of publish daemons | ||
| - **Message rates**: Publishing vs consuming rates | ||
| - **Connection status**: Watch for disconnections | ||
|
|
||
| RabbitMQ Management UI (if enabled): `http://localhost:15672` | ||
|
Comment on lines
+143
to
+150
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So Edge actually has our RabbitMQ instance configured with Prometheus metrics. I don't remember the details of how we turned that on (probably some config file), but we ultimately proxy those through Caddy and they end up at |
||
|
|
||
| ## Next Steps | ||
|
|
||
| After configuring AMQP: | ||
|
|
||
| 1. Start the server: `yarn start` | ||
| 2. Start the publish daemon: `yarn publish-daemon` | ||
| 3. Monitor logs: | ||
| - `/var/log/pushServer.log` | ||
| - `/var/log/publishDaemon.log` | ||
| 4. Test the connection using the demo script: `yarn demo` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is important, since any version will do. We just need the AMQP 0.9.1 protocol, and really any implementation would be fine. Doesn't even need to be RabbitMQ, and it certainly doesn't need to be the Dockerized version.
Could we have the agent tone this down a bit? The docs mention this exact version RabbitMQ version many, many times, and it's really not necessary to be so picky.