Building a logging pipeline with open-source tools

Speaker: Iñigo Ortiz de Urbin

Abstract

In this talk we will have a look at the general architecture of the logging pipeline built at the RIPE NCC to stream, store and analyse the logs created by the systems powering the RIPE Atlas project backend. We will also look into how to adapt the open source tools involved in our logging sink to process your own logs. The ELK stack (Elasticsearch, Logstash, Kibana) is part of them.

Slides

Biography

Iñigo is a Systems Engineer working for the RIPE NCC in the team responsible for managing the RIPE Atlas and Routing Information Service backends as well as several authoritative DNS clusters (AS112, AS25152, AS197000). He has a taste for DNS, BGP, data processing solutions, python and automation. For the last year he has implemented a logging pipeline to help with the operations and increase the visibility of the systems. In his free time he enjoys tinkering with OpenBSD, cooking while listening to classic jazz and caring about matters such as privacy, free speech and politics.