Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Edge Push Server Documentation Index

This file serves as the central index for all documentation in the Edge Push Server project.

## Documentation Structure

### Project Setup and Overview

#### `README.md`

- **When to read**: First time setting up the project, understanding basic server setup and deployment
- **Summary**: Project overview, setup instructions including AMQP/RabbitMQ 3.12 Docker configuration, PM2 management, and deployment procedures

### Configuration Guides

#### `docs/guides/amqp-configuration.md`

- **When to read**: Before running `yarn start` for the first time, when setting up development environment, or troubleshooting message queue issues
- **Summary**: Complete guide for configuring the AMQP client connection required by the push server, including RabbitMQ 3.12 Docker setup, connection string format, security considerations, and troubleshooting tips

### API Documentation

#### `docs/guides/marketing-api.md`

- **When to read**: When implementing marketing campaigns, sending location-based push notifications, or integrating with the marketing task queue system
- **Summary**: Complete documentation for the Marketing API endpoints including authentication, location targeting, task queue management, progress tracking, and integration examples. Covers device filtering, error handling, and best practices for large-scale push notification campaigns.

### Technical References

#### `docs/guides/marketing-database-schema.md`

- **When to read**: When working on marketing system internals, database maintenance, or understanding the task queue architecture
- **Summary**: Technical documentation of database schema changes for the Marketing API, including ApiKey updates, MarketingTask structure, CouchDB views, migration considerations, and monitoring queries. Essential for developers working on the marketing system backend.

### Migration Guides

#### `docs/guides/marketing-api-migration.md`

- **When to read**: When upgrading an existing Edge Push Server installation to include Marketing API support
- **Summary**: Step-by-step migration guide covering code updates, database migration, API key configuration, daemon setup, and verification procedures. Includes troubleshooting tips, rollback procedures, and post-migration tasks for existing installations.

### Additional Resources

#### `docs/demo.ts`

- **When to read**: When learning how to integrate with the v2 API, testing push notifications
- **Summary**: Example TypeScript code demonstrating how to use the Edge Push Server v2 API

#### `docs/logrotate`

- **When to read**: When setting up production server, configuring log management
- **Summary**: Log rotation configuration for managing server logs in production

## Quick Start

1. Read `README.md` for project setup
2. Configure AMQP by following `docs/guides/amqp-configuration.md`
3. Set up your `pushServerConfig.json` with database and AMQP credentials
4. Run `yarn install` and `yarn prepare`
5. Start the server with `yarn start`

## Architecture Overview

The Edge Push Server consists of:

- HTTP API server for device registration and notification triggers
- AMQP message queue for reliable message delivery
- Background daemons for processing notifications, price changes, and confirmations
- CouchDB for storing device registrations and settings
- Firebase Admin SDK for sending push notifications
44 changes: 42 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,51 @@

This server sends push notifications to Edge client apps. It contains an HTTP server that clients can use to register for notifications, and a background process that checks for price changes and actually sends the messages.

The docs folder has can find [an example of how to use the v2 API](./docs/demo.ts).
## Features

- **Device Registration API**: Register devices for push notifications
- **Event-based Notifications**: Price changes, transaction confirmations, balance alerts
- **Marketing API**: Location-based marketing campaigns with async task processing
- **Background Processing**: Reliable message delivery via AMQP queues

## Documentation

- **[Marketing API Guide](./docs/guides/marketing-api.md)** - Complete guide for location-based push campaigns
- **[API Integration Example](./docs/demo.ts)** - TypeScript example for v2 API integration
- **[AMQP Configuration](./docs/guides/amqp-configuration.md)** - Detailed message queue setup
- **[Migration Guides](./docs/guides/)** - For upgrading existing installations

## Setup

This server requires a working copies of Node.js, Yarn, PM2, and CouchDB. We also recommend using Caddy to terminate SSL connections.
This server requires a working copies of Node.js, Yarn, PM2, CouchDB, and RabbitMQ 3.12 (via Docker). We also recommend using Caddy to terminate SSL connections.

### Configure AMQP Message Queue

The push server uses AMQP (RabbitMQ) for reliable message delivery between the HTTP server and push notification daemons. Before running `yarn start`, you must:

1. **Start RabbitMQ 3.12 using Docker**:

```bash
docker run -d --name rabbitmq \
-p 5672:5672 \
-p 15672:15672 \
-e RABBITMQ_DEFAULT_USER=guest \
-e RABBITMQ_DEFAULT_PASS=guest \
rabbitmq:3.12-management
```

2. **Create a configuration file** `pushServerConfig.json` in the project root:

```json
{
"listenHost": "127.0.0.1",
"listenPort": 8008,
"amqpUri": "amqp://guest:guest@localhost:5672",
"couchUri": "http://username:password@localhost:5984"
}
```

For detailed AMQP configuration instructions, troubleshooting, and security considerations, see [docs/guides/amqp-configuration.md](./docs/guides/amqp-configuration.md).

### Set up logging

Expand Down
161 changes: 161 additions & 0 deletions docs/guides/amqp-configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# AMQP Client Configuration Guide

## Overview

The Edge Push Server uses AMQP (Advanced Message Queuing Protocol) for reliable message delivery between the HTTP server and the push notification daemon. This guide explains how to configure the AMQP client before running the server.

## What is AMQP Used For?

The AMQP client serves as the messaging backbone for the push notification system:

1. **Message Queue**: The HTTP server enqueues push notification requests into a RabbitMQ/AMQP queue named "messages"
2. **Decoupling**: Separates the HTTP API from the actual push notification delivery process
3. **Reliability**: Ensures messages aren't lost if the push daemon is temporarily down
4. **Load Management**: The queue prefetch is set to 50 messages to prevent overwhelming the push daemon

## Architecture Flow

```
HTTP Server → AMQP Queue ("messages") → Publish Daemon → Firebase/Push Services
```

## Configuration Requirements

### 1. Create Configuration File

Before running `yarn start`, create a `pushServerConfig.json` file in the project root:

```json
{
"listenHost": "127.0.0.1",
"listenPort": 8008,
"amqpUri": "amqp://username:password@localhost:5672",
"couchUri": "http://username:password@localhost:5984",
"currentCluster": "production"
}
```

### 2. AMQP URI Format

The `amqpUri` follows the standard AMQP connection string format:

```
amqp://[username[:password]@]hostname[:port][/vhost]
```

Examples:

- Local development (Docker): `amqp://guest:guest@localhost:5672`
- Production with vhost: `amqp://edgeuser:securepass@rabbitmq.example.com:5672/edge`
- CloudAMQP: `amqp://user:pass@hostname.cloudamqp.com/instance`

### 3. Required AMQP Server Setup

Before starting the Edge Push Server, ensure you have:

1. **RabbitMQ Server version 3.12** (or compatible AMQP broker) running via Docker
2. **User credentials** with permissions to:
- Create queues
- Publish messages
- Consume messages
- Set prefetch count
3. **Network access** to the AMQP server on port 5672

### 4. Queue Configuration

The server automatically creates a queue named "messages" with:

- **Prefetch limit**: 50 messages (prevents memory overflow)
- **Manual acknowledgment**: Messages are only removed after successful processing
- **Durable**: Queue persists through server restarts (implementation dependent)

## RabbitMQ Setup (Docker)

**Important**: The Edge Push Server requires RabbitMQ version 3.12 with management interface.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is important, since any version will do. We just need the AMQP 0.9.1 protocol, and really any implementation would be fine. Doesn't even need to be RabbitMQ, and it certainly doesn't need to be the Dockerized version.

Could we have the agent tone this down a bit? The docs mention this exact version RabbitMQ version many, many times, and it's really not necessary to be so picky.


Run the following Docker command to set up RabbitMQ:

```bash
docker run -d --name rabbitmq \
-p 5672:5672 \
-p 15672:15672 \
-e RABBITMQ_DEFAULT_USER=guest \
-e RABBITMQ_DEFAULT_PASS=guest \
rabbitmq:3.12-management
```

This command:

- Uses RabbitMQ version 3.12 with management interface
- Exposes port 5672 for AMQP connections
- Exposes port 15672 for the web management UI
- Sets default credentials (guest/guest)

### Cloud Services Alternative

- **CloudAMQP**: Managed RabbitMQ hosting
- **Amazon MQ**: AWS managed message broker
- **Azure Service Bus**: Alternative AMQP-compatible service

## Troubleshooting

### Connection Errors

If you see connection errors when starting the server:

1. **Check AMQP server is running**:

```bash
# For Docker RabbitMQ
docker ps | grep rabbitmq
docker logs rabbitmq
```

2. **Verify credentials**:

```bash
# Test connection
curl -i -u username:password http://localhost:15672/api/overview
```

3. **Check firewall/network**:
```bash
telnet localhost 5672
```

### Common Issues

- **"Connection refused"**: AMQP server not running or wrong port
- **"Authentication failed"**: Incorrect username/password
- **"Access refused"**: User lacks necessary permissions
- **"Channel closed"**: Often indicates permission issues or resource limits

## Security Considerations

1. **Never commit** `pushServerConfig.json` with real credentials
2. **Use strong passwords** for production AMQP instances
3. **Enable TLS** for production: `amqps://` instead of `amqp://`
4. **Limit network access** to AMQP ports using firewall rules
5. **Use separate vhosts** for different environments (dev/staging/prod)

## Monitoring

Monitor your AMQP queue health:

- **Queue depth**: Messages waiting to be processed
- **Consumer count**: Should match number of publish daemons
- **Message rates**: Publishing vs consuming rates
- **Connection status**: Watch for disconnections

RabbitMQ Management UI (if enabled): `http://localhost:15672`
Comment on lines +143 to +150

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So Edge actually has our RabbitMQ instance configured with Prometheus metrics. I don't remember the details of how we turned that on (probably some config file), but we ultimately proxy those through Caddy and they end up at https://push-x.edge.app:443/metrics/rabbitmq-somesecretstuff. This allows us to see the queue depth very nicely in our normal Grafana dashboard.


## Next Steps

After configuring AMQP:

1. Start the server: `yarn start`
2. Start the publish daemon: `yarn publish-daemon`
3. Monitor logs:
- `/var/log/pushServer.log`
- `/var/log/publishDaemon.log`
4. Test the connection using the demo script: `yarn demo`
Loading
Loading