Log Aggregation: Qryn

Qryn is a Loki compatible log (and metrics) aggregation system using ClickHouse under the hood. Promtail is used to feed logs into qryn. Finally we configure Grafana from our Monitoring stack to use qryn as a data source.

Github: https://github.com/metrico/qryn
Documentation: https://qryn.metrico.in

Architecture Decision Record


  • Simple monolithic architecture (qryn HTTP server + ClickHouse as data store)
  • Fully Loki / LogQL compatible
  • Also supports other data sources (we don't make use of this currently)
  • ClickHouse as a backend equals excellent performance and scalability
  • ClickHouse can be configured to use S3 for storage for increased durability and "bottomless" storage


  • Requires extra configuration for ClickHouse
  • Not as widely used as Loki

Alternatives Considered


At Mynewsdesk we use Loki for our in house version of the platform. It has served us quite well but architecturally it has significantly more moving parts than Qryn which means it becomes harder to reason about. It's also dependent on S3 if running in it's "scalable" mode which makes it harder to create a good "it just works" experience for our open source platform.

We have also run into the odd performance issue with Loki, especially when it comes to querying large amounts of data. Loki version 2.8.0 introduced a new TSBD Index which promises a 4x performance improvement which may bring performance on par with ClickHouse. We have not yet had the chance to test this out.


You can tail logs from applications via k logs <application-name>. You can also filter the logs based on type of deployment by passing the types as additional arguments to the command. Eg. k logs <application-name> web sidekiq.

To do advanced search, filtering and aggregation of logs, you can use the Explore view in Grafana and selecting the Logging datasource. Run k logs:search <application-name> to quickly open the Grafana log UI for a specific application.

Read more on how to work with logs in the Grafana UI in their Logs in Explore documentation.