based on user reports and queries over the last 24 hours
ClickHouse outage statistics
- Official Website at https://clickhouse.com;
- Community Slack at https://clickhouse.com/slack;
- GitHub Issues at https://github.com/ClickHouse/ClickHouse/issues;
- Documentation at https://clickhouse.com/docs;
- Community Forum at https://github.com/ClickHouse/ClickHouse/discussions;
- Cloud Support at https://clickhouse.cloud/support (for cloud customers).
- Official Website - https://clickhouse.com;
- Documentation - https://clickhouse.com/docs;
- GitHub - https://github.com/ClickHouse/ClickHouse;
- Blog - https://clickhouse.com/blog;
- X (Twitter) - https://twitter.com/ClickHouseDB;
- YouTube - https://www.youtube.com/c/ClickHouseDB;
- LinkedIn - https://www.linkedin.com/company/clickhouse;
- Slack Community - https://clickhouse.com/slack;
- Cloud Status - https://status.clickhouse.cloud;
- Release Notes - https://clickhouse.com/docs/en/whats-new/changelog.
Service disruptions manifest as ClickHouse server not responding, queries timing out, connection refused errors, cluster unreachable, or complete database unavailable.
- Check ClickHouse server status: `systemctl status clickhouse-server`;
- Review server logs: `/var/log/clickhouse-server/clickhouse-server.log`;
- Verify server process running: `ps aux | grep clickhouse`;
- Check port 8123 (HTTP) or 9000 (native) accessible;
- Restart server: `sudo systemctl restart clickhouse-server`;
- Verify configuration file syntax correct;
- Check disk space available;
- Review resource usage (CPU/RAM);
- Test local connection: `clickhouse-client`;
- Monitor ClickHouse Cloud status if using managed service.
Performance issues include slow SELECT queries, high memory usage, timeouts, INSERT delays, or merge operations blocking.
- Check query execution plan with EXPLAIN;
- Review running queries: `SHOW PROCESSLIST`;
- Kill long-running queries if needed;
- Verify table partitioning configured;
- Check primary key and indexes optimal;
- Review merge tree settings;
- Monitor system.query_log table;
- Optimize table manually if needed;
- Check background merge activity;
- Review max_threads and max_memory settings.
Connection problems include authentication failing, "Connection refused", network unreachable, SSL errors, or user permission denied.
- Verify credentials correct;
- Check user permissions granted;
- Review users.xml configuration;
- Test connection with clickhouse-client;
- Verify firewall allows connections;
- Check listen_host configuration;
- Review network security settings;
- Test with different client/driver;
- Verify SSL certificates if enabled;
- Check access control policies.
INSERT problems include "Table doesn't exist" errors, data type mismatches, quota exceeded, read-only mode, or replication lag.
- Verify table exists in database;
- Check column data types match;
- Review INSERT query syntax;
- Check user write permissions;
- Verify quota not exceeded;
- Check table not in readonly mode;
- Review replication queue status;
- Verify partition key values valid;
- Check disk space sufficient;
- Review server error logs.
Cluster issues include replica unavailable, ZooKeeper connection lost, distributed table errors, shard unreachable, or replication broken.
- Check all cluster nodes online;
- Verify ZooKeeper service running;
- Review cluster configuration correct;
- Test connection to each shard;
- Check replica status in system.replicas;
- Verify network between nodes stable;
- Review distributed_ddl_queue table;
- Check replication queue not stuck;
- Verify ZooKeeper quorum healthy;
- Review remote_servers configuration.
Resource problems include OOM errors, server crashes, "Memory limit exceeded", high CPU usage, or swap thrashing.
- Check memory usage: `clickhouse-client -q "SELECT * FROM system.metrics"`;
- Review max_memory_usage settings;
- Kill memory-heavy queries;
- Optimize queries to use less memory;
- Increase server RAM if needed;
- Check background merges resource usage;
- Review max_bytes_before_external_* settings;
- Monitor system.asynchronous_metrics;
- Adjust max_server_memory_usage;
- Check for memory leaks in logs.
Table management issues include table corruption, partition not dropping, ALTER failing, metadata errors, or detached parts.
- Check table structure: `DESCRIBE table`;
- Verify table engine configuration;
- Review detached parts directory;
- Attempt table repair if needed;
- Check partition expressions valid;
- Review ALTER TABLE operation logs;
- Verify no locks blocking operations;
- Check mutations status;
- Review system.parts table;
- Backup before major changes.
Backup problems include snapshot creation errors, restore failing, data corruption, insufficient space, or clickhouse-backup issues.
- Use clickhouse-backup tool properly;
- Check backup storage sufficient;
- Verify backup configuration correct;
- Test restore on separate instance;
- Review backup logs for errors;
- Check S3/filesystem permissions;
- Verify incremental backup chain;
- Use FREEZE TABLE for snapshots;
- Check backup retention policy;
- Test disaster recovery procedure.
Materialized view problems include MV not populating, data lag, refresh errors, target table issues, or query failures.
- Check MV definition correct;
- Verify source table receiving data;
- Review MV target table exists;
- Check for errors in system.query_log;
- Drop and recreate MV if needed;
- Verify POPULATE used if required;
- Check resource limits not blocking;
- Review MV query performance;
- Monitor MV refresh frequency;
- Check dependencies resolved.
Cloud-specific issues include service unavailable, scaling problems, API errors, billing issues, or connectivity to cloud instance.
- Check ClickHouse Cloud status page;
- Verify cloud credentials valid;
- Review service tier limits;
- Check API quota not exceeded;
- Test connection string correct;
- Verify IP whitelist configured;
- Review cloud console for alerts;
- Check billing account active;
- Monitor auto-scaling behavior;
- Contact cloud support team.
Integration problems include JDBC/ODBC errors, Python driver issues, API failures, incompatible versions, or connection pooling problems.
- Update driver to latest version;
- Verify driver compatibility with server;
- Check connection parameters correct;
- Review driver documentation;
- Test with official clients first;
- Check SSL/TLS settings match;
- Verify connection pool configuration;
- Review driver error messages;
- Test with simple queries first;
- Check for known driver bugs.
Configuration problems include invalid settings, server won't start, config merge errors, profile issues, or setting overrides failing.
- Verify XML syntax in config files;
- Check config.xml and users.xml valid;
- Review configuration merge priority;
- Test config with --config-file option;
- Check included config files exist;
- Verify settings not conflicting;
- Review system.settings table;
- Check server startup logs;
- Validate changes before restart;
- Keep backup of working configs.
ClickHouse
Your message will be published in about
5 minutes
Service administration will see your message