By James Leffel & Co.
Central Logging Service for Distributed Applications
With the release of our great new presentation layer for the Goettingen Digitization Centre we faced new problems. The Application consists of many different separate services (and servers). For example a Symfony frontend application, separate applications to deliver and process IIIF manifests and images. One service is responsible to get and process newly scanned works, one genereates PDF files, and so on.
Each service may fail and behave somehow unexpected, but these errors are not visible to the user, but logged inside the application.
Slowly we get to the point, but one may imagine, that looking for errors in each of these services and handling with different logging formats is a lot of work and very error prone.
So, using a central logging server for all these applications is the solution to our problem. We decided to use Graylog2 that is quite easy to setup in our docker-based environment. It’s nothing more than
docker run --name mongo -d mongo:3 docker run --name elasticsearch \ -e "http.host=0.0.0.0" -e "xpack.security.enabled=false" \ -d docker.elastic.co/elasticsearch/elasticsearch:5.6.6 docker run --link mongo --link elasticsearch \ -p 9000:9000 -p 8441:8441/udp -p 12201:12201 -p 514:514 -p 53160:53160/udp -p 7946:7946/udp -p 8084:8084 \ -e GRAYLOG_WEB_ENDPOINT_URI="https://logging.example.com:9000/api" \ -d graylog/graylog:2.4.3-1
and the service is up.
Next up, we configure our Symfony Apps to use the Gelf LogHandler (the default format for logging to Graylog2).
composer require graylog2/gelf-php adds the formatter and adding a service and add the handler to the monolog configuration like that:
monolog.gelf_handler: class: Monolog\Handler\GelfHandler arguments: ["@gelf.publisher", 200] gelf.validator: class: Gelf\MessageValidator gelf.transport: class: Gelf\Transport\TcpTransport arguments: ['%env(LOG_DOMAIN)%', '%env(LOG_PORT)%'] gelf.publisher: class: Gelf\Publisher arguments: ["@gelf.transport", "@gelf.validator"]
and finally the monolog configuration in
monolog: handlers: main: type: fingers_crossed level: info handler: gelf gelf: type: service id: monolog.gelf_handler level: debug
With these small steps we have a fully working, centralized logging for our services (at least our Symfony applications).