​

Platform Monitoring & Observability

​

Monitoring Overview

┌─────────────────────────────────────────────────────────┐
│                 Centralized Monitoring                  │
│                                                         │
│  ┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐  │
│  │ Metrics │   │  Logs   │   │ Traces  │   │ Alerts  │  │
│  └─────────┘   └─────────┘   └─────────┘   └─────────┘  │
└─────────────────────────────────────────────────────────┘
               ▲           ▲           ▲           ▲
               │           │           │           │
┌──────────────┴───┐ ┌─────┴──────────┐ ┌─────────┴────────┐ ┌──────────┴─────────┐
│   Ollama Service  │ │ Flowise Service │ │ Langchain Service │ │    n8n Service    │
└──────────────────┘ └────────────────┘ └──────────────────┘ └────────────────────┘

​

Monitoring Dashboards

​

Accessing Monitoring Dashboards

• Production Environment: https://monitor.moodmnky.com
• Development Environment: http://localhost:3030

​

Available Dashboards

​

Key Metrics

​

Service Health Metrics

​

Resource Utilization Metrics

​

Service-Specific Metrics

​

Ollama Service

• Model loading time
• Inference latency
• Token generation rate
• Cache hit ratio
• Concurrent requests

​

Flowise Service

• Workflow execution time
• Node processing time
• Queue length
• Memory consumption per workflow
• Error rates by node type

​

Langchain Service

• Document processing time
• Embedding generation rate
• Vector search latency
• Retrieval accuracy
• Memory usage by collection

​

n8n Service

• Workflow execution time
• Queue length
• Node execution metrics
• External service latency
• Error rates by workflow

​

Logging System

​

Log Collection

• Log Format: JSON structured logs
• Log Levels: ERROR, WARN, INFO, DEBUG, TRACE
• Contextual Information: Request ID, service name, timestamp, user ID (when available)
• Centralized Storage: Logs are aggregated in a centralized logging system

​

Accessing Logs

​

Centralized Logging UI

• Production: https://logs.moodmnky.com
• Development: http://localhost:5601

​

Service-Specific Logs

# Ollama logs
docker logs ollama-service

# Flowise logs
tail -f ./flowise/logs/app.log

# Langchain logs
tail -f ./langchain/logs/server.log

# n8n logs
tail -f ~/.n8n/logs/n8n.log

​

Log Querying

level:ERROR AND timestamp:[now-1h TO now]

requestId:"req_12345abcde" 

​

Tracing System

​

Trace Components

• Trace ID: Unique identifier for each request flow
• Spans: Individual operations within a trace
• Span Attributes: Context data for each span
• Service Map: Visual representation of service dependencies

​

Accessing Traces

• Production: https://traces.moodmnky.com
• Development: http://localhost:16686

​

Alerting System

​

Alert Configuration

​

Alert Notifications

• Email: Sent to registered developer and operations emails
• Slack: Posted to the #service-alerts channel
• SMS: Sent to on-call personnel for critical alerts
• Dashboard: Visible on monitoring dashboards

​

Monitoring API

import axios from 'axios';

// Configuration
const baseUrl = 'https://api.moodmnky.com/monitoring';
const apiKey = 'your_api_key';

// Get service health status
async function getServiceHealth(serviceName: string) {
  try {
    const response = await axios.get(
      `${baseUrl}/services/${serviceName}/health`,
      {
        headers: {
          'Authorization': `Bearer ${apiKey}`
        }
      }
    );
    
    return response.data;
  } catch (error) {
    console.error('Error retrieving service health:', error);
    throw error;
  }
}

// Get recent errors for a service
async function getServiceErrors(serviceName: string, limit = 20) {
  try {
    const response = await axios.get(
      `${baseUrl}/services/${serviceName}/errors?limit=${limit}`,
      {
        headers: {
          'Authorization': `Bearer ${apiKey}`
        }
      }
    );
    
    return response.data;
  } catch (error) {
    console.error('Error retrieving service errors:', error);
    throw error;
  }
}

​

Health Check Endpoints

​

Troubleshooting Guide

​

Common Issues

​

Service Unreachable

1. Check service health endpoint
2. Verify network connectivity
3. Check for recent deployments
4. Review error logs
5. Check resource utilization

​

High Latency

1. Monitor request volume
2. Check database performance
3. Review resource utilization
4. Check for long-running processes
5. Identify slow dependencies

​

Increased Error Rate

1. Identify error patterns
2. Check for recent code changes
3. Review dependency health
4. Check for rate limiting issues
5. Verify configuration settings

​

Diagnostic Commands

# Check service status
curl http://localhost:{port}/health

# View resource usage
docker stats

# Check network connectivity
curl -v http://localhost:{port}

# View recent logs
tail -n 100 ./logs/service.log

​

Best Practices

​

Monitoring Implementation

1. Standard Metrics: Implement the same core metrics across all services
2. Contextual Logging: Include request IDs in all logs
3. Alert Tuning: Regularly review and adjust alert thresholds
4. Correlation: Correlate metrics, logs, and traces
5. Historical Data: Maintain historical data for trend analysis

​

Operational Procedures

1. Regular Review: Schedule monitoring review sessions
2. Runbooks: Create standard procedures for common issues
3. Post-Incident Analysis: Conduct post-mortems after incidents
4. Continuous Improvement: Refine monitoring based on incidents
5. Documentation: Keep monitoring documentation updated

​

Support Resources

• Monitoring System Documentation
• Service Status Page
• Developer Portal
• Support Email