Troubleshooting

This guide covers common issues encountered when deploying and running Dharini.

Docker Issues

Containers Won’t Start

Problem: Services fail to start or immediately exit

Check service status:

docker ps -a
docker logs backend
docker logs frontend

Common causes:

1. Port conflicts - Another service using ports 3000, 4000, 5432

   # Check what's using a port
   sudo lsof -i :4000
   
   # Change ports in docker-compose.yml if needed

2. Environment variables missing

   # Verify .env file exists and has required variables
   cat .env | grep DB_HOST

3. Docker daemon not running

   sudo systemctl status docker
   sudo systemctl start docker

Backend Container Restarting

Problem: Backend container in restart loop

Check logs:

docker logs backend --tail 100

Common issues:

1. Database connection failed

   • Verify DB_HOST, DB_PORT, DB_USERNAME, DB_PASSWORD in .env
   • For RDS, ensure SSL is enabled (automatic in production)
   • Check RDS security group allows EC2 access

2. Missing environment variables

   • Check backend logs for “undefined” errors
   • Ensure all required variables are set

3. Redis connection failed

   • Verify Redis container is running: docker ps | grep redis
   • Check REDIS_HOST=redis in .env

Frontend Container Unhealthy

Problem: Frontend shows as “unhealthy” status

This is often normal during:

• Initial build
• After code changes
• Server Action cache mismatches

Solutions:

1. Wait 1-2 minutes - Health check may pass after warm-up

2. Rebuild frontend:

   docker compose --profile production restart frontend

3. Clear browser cache - Hard refresh (Ctrl+Shift+R)

4. Full rebuild:

   docker compose --profile production down
   docker compose --profile production up -d --build

Database Issues

Cannot Connect to RDS

Problem: “no pg_hba.conf entry for host” or connection timeout

Solutions:

1. Use SSL for RDS connections:

   PGSSLMODE=require psql -h your-rds-endpoint -U postgres -d postgres

2. Check security group:

   • RDS security group must allow port 5432 from EC2
   • Verify EC2 security group ID or private IP is allowed

3. Verify RDS endpoint:

   # Test DNS resolution
   nslookup your-database-instance.xxxxxxxxx.region.rds.amazonaws.com
   
   # Test connectivity
   nc -zv your-database-instance.xxxxxxxxx.region.rds.amazonaws.com 5432

4. Check credentials:

   • Verify username and password are correct
   • Try connecting from another tool (pgAdmin, DBeaver)

PostGIS Extension Missing

Problem: Migrations fail with “type ‘geometry’ does not exist”

Solution:

Connect to RDS and install PostGIS:

CREATE EXTENSION IF NOT EXISTS postgis;
CREATE EXTENSION IF NOT EXISTS postgis_topology;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

Migration Failures

Problem: npm run migration:run fails

Check:

1. View detailed error:

   docker logs backend -f

2. Verify database connection:

   docker exec -it backend env | grep DB_

3. Check migration table:

   SELECT * FROM migrations;

4. Force rerun (if migration was partially applied):

   DELETE FROM migrations WHERE name = 'FailedMigration';

Network and Nginx Issues

502 Bad Gateway

Problem: Nginx returns 502 error

Causes:

1. Backend not running:

   docker ps | grep backend
   docker logs backend

2. Wrong upstream IP in Nginx config:

   # Verify EC2 private IP
   hostname -I
   
   # Update nginx config to match
   sudo nano /etc/nginx/sites-available/dharini-prod

3. Firewall blocking ports:

   # Check if ports are listening
   sudo netstat -tulpn | grep -E ':(3000|4000|3001)'

504 Gateway Timeout

Problem: Requests time out

Solutions:

1. Increase Nginx timeout:

   location /api/ {
       proxy_read_timeout 300;
       proxy_connect_timeout 300;
       proxy_send_timeout 300;
       # ... other settings
   }

2. Check backend performance:

   docker stats backend

SSL Certificate Errors

Problem: Certificate not working or expired

Solutions:

1. Test renewal:

   sudo certbot renew --dry-run

2. Force renewal:

   sudo certbot renew --force-renewal

3. Regenerate certificate:

   sudo certbot --nginx -d your-domain.com

API 404 Errors

Problem: /api/ requests return 404

Check:

1. Nginx path configuration:

   # Strips /api/ prefix
   location /api/ {
       proxy_pass http://YOUR_PRIVATE_IP:4000/;  # Trailing slash important!
   }

2. Backend routes:

   • Backend should define routes as /users, not /api/users
   • Nginx strips /api/ before forwarding

3. Test directly:

   # Bypass Nginx - test backend directly
   curl http://localhost:4000/users

Performance Issues

High CPU Usage

Check resource usage:

docker stats

Common causes:

1. Database queries - Check slow query log
2. Infinite loops in background jobs
3. Memory leaks - Monitor over time

Solutions:

1. Scale EC2 instance - Upgrade to larger instance type
2. Optimize queries - Add indexes, use EXPLAIN ANALYZE
3. Restart services:

   docker compose --profile production restart

Out of Memory

Problem: Services crash with OOM errors

Check memory:

free -h
docker stats

Solutions:

1. Increase EC2 instance memory

2. Add swap space (temporary):

   sudo fallocate -l 2G /swapfile
   sudo chmod 600 /swapfile
   sudo mkswap /swapfile
   sudo swapon /swapfile

3. Reduce Docker memory limits in docker-compose.yml

Slow API Responses

Debug:

1. Check database performance:

   SELECT * FROM pg_stat_statements ORDER BY mean_time DESC LIMIT 10;

2. Monitor RDS metrics in CloudWatch

3. Check Redis connectivity:

   docker exec -it redis redis-cli ping

Application Issues

Login Not Working

Problem: Cannot log in with phone/OTP

Check:

1. Super admin user exists:

   SELECT * FROM users WHERE designation = 'super-admin';

2. OTP service configured:

   • Check SENDGRID_API_KEY or GMAIL_USER/GMAIL_PASS
   • View backend logs for email send errors

3. Database connection:

   docker logs backend | grep -i "database"

File Upload Failing

Problem: Cannot upload images or files

Check:

1. S3 credentials:

   # Test from backend container
   docker exec -it backend env | grep S3_

2. S3 bucket permissions:

   • Verify IAM user has PutObject permission
   • Check bucket policy

3. File size limits:

   • Nginx: client_max_body_size
   • Application: MAX_FILE_ALLOW_UPLOAD

4. Backend logs:

   docker logs backend | grep -i "upload"

Maps Not Loading

Problem: Maps or geospatial features not working

Check:

1. PostGIS installed:

   SELECT PostGIS_version();

2. Spatial data:

   SELECT id, ST_AsText(location) FROM sites LIMIT 5;

3. Frontend map library loaded (check browser console)

Deployment Issues

Git Pull Fails

Problem: Cannot pull latest code

Solutions:

1. Stash local changes:

   git stash
   git pull origin demo
   git stash pop

2. Force pull (discards local changes):

   git fetch origin
   git reset --hard origin/demo

Docker Build Fails

Problem: docker compose build fails

Common issues:

1. Disk space full:

   df -h
   docker system prune -a

2. Network issues downloading packages:

   # Retry with verbose output
   docker compose build --no-cache --progress=plain

3. Dockerfile errors - Check backend/frontend Dockerfile

Monitoring and Logs

View All Logs

# All services
docker compose logs -f

# Specific service
docker logs -f backend
docker logs -f frontend --tail 100

# Since specific time
docker logs backend --since 30m

System Resources

# Disk usage
df -h
du -sh /var/lib/docker

# Memory
free -h

# Docker stats
docker stats

# Processes
top
htop  # if installed

Network Debugging

# Test port connectivity
nc -zv your-domain.com 443

# DNS resolution
nslookup your-domain.com

# Trace route
traceroute your-domain.com

# Check listening ports
sudo netstat -tulpn

Getting Help

If issues persist:

1. Check logs for specific error messages
2. Search issues on GitHub: https://github.com/dsih-artpark/dharini/issues
3. Create new issue with:
   • Error logs
   • Environment details (OS, Docker version)
   • Steps to reproduce
4. Contact the development team

Preventive Measures

1. Regular backups - Database and configuration
2. Monitor disk space - Set up alerts
3. Update regularly - Pull latest code weekly
4. Test in staging before production updates
5. Document changes to configuration
6. Monitor logs for errors and warnings

Quick Fixes Checklist

When something breaks, try these in order:

1. ☐ Check Docker containers: docker ps
2. ☐ View logs: docker logs backend
3. ☐ Restart services: docker compose restart
4. ☐ Check .env file variables
5. ☐ Verify database connection
6. ☐ Test Nginx config: sudo nginx -t
7. ☐ Check security groups (AWS)
8. ☐ Review recent changes: git log
9. ☐ Rebuild if needed: docker compose up -d --build
10. ☐ Check disk space: df -h