Systemd does not see all shutdowns as failures

[djsly]

Bug report

System info:

telegraf --version
Telegraf v1.2.1 (git: release-1.2 3b6ffb344e5c03c1595d862282a6823ecb438cff)

Linux 3.10.0-327.18.2.el7.x86_64 x86_64 x86_64 GNU/Linux

cat /etc/system-release
CentOS Linux release 7.2.1511 (Core) 

Steps to reproduce:

1. systemctl stop systemd-journald.socket
2. systemctl stop systemd-journald.service
3. systemctl start systemd-journald.service

wait for telegraf to try to write a log file. (start with debug to accelerate)

Expected behavior:

Telegraf process stays up Or systemd restart it properly

Actual behavior:

Telegraf dies and never comes back

Additional info:

[root@<hostname> ~]# systemctl status telegraf -l
● telegraf.service - The plugin-driven server agent for reporting metrics into InfluxDB
   Loaded: loaded (/usr/lib/systemd/system/telegraf.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Tue 2017-04-25 10:33:40 EDT; 14min ago
     Docs: https://github.com/influxdata/telegraf
 Main PID: 7857 (code=killed, signal=PIPE)

This occurs when we restart systemd-journald, which cause a crash/stop of telegraph seen as a STDPIPE

See: https://bugs.freedesktop.org/show_bug.cgi?id=84923#c9
With affected services like Consul: hashicorp/consul#1688
and Prometheus: https://github.com/voxpupuli/puppet-prometheus/pull/3/files

[phemmer]

In addition to the PR currently up (#2716), another proposal which would help with this would be to use syslog. If logs were sent to syslog we could close STDOUT/STDERR, and would no longer get the SIGPIPE.