EMFILE: Too Many Open Files, and Why Raising the Limit Is Usually Wrong

The short answer

Error: EMFILE: too many open files, open '/app/uploads/tmp-8821'

Find the real count before you touch any limit:

# how many descriptors does the process actually hold
ls /proc/$(pgrep -f "node server.js")/fd | wc -l

# what are they
ls -l /proc/$(pgrep -f "node server.js")/fd | awk '{print $NF}' | sort | uniq -c | sort -rn | head

# the limits currently applied to that running process
cat /proc/$(pgrep -f "node server.js")/limits | grep "open files"

If that list is dominated by socket:[...] entries you have a connection leak, not a file leak. If it is dominated by one path repeated hundreds of times, you have a handle you never closed.

Tested on Node 22.14, Linux 6.8, systemd 255.

Raising ulimit -n is the right move when your service legitimately needs more than 1024 descriptors, which is most network services. It is the wrong move when the count grows without bound, because you are buying time before the same crash.

Why the limit exists at all

Every open file, every socket, every pipe and every epoll instance costs a file descriptor. The kernel tracks them per process in a table, and the soft limit is what stops one buggy process exhausting the system.

There are three ceilings, and people usually adjust the wrong one:

A shell ulimit -n 65535 changes the soft limit for that shell and anything it spawns. It does nothing for a service started by systemd, which is the mistake I see most often. Systemd services ignore your shell entirely and take LimitNOFILE from the unit file.

[Service]
LimitNOFILE=65535

Then systemctl daemon-reload and restart the service. Check it landed by reading /proc/<pid>/limits rather than trusting the config, because a running process keeps the limits it started with.

The four causes, in the order I check them

1. Sockets that are never closed

This is the common one, and it is the reason the descriptor count climbs steadily rather than spiking.

ls -l /proc/<pid>/fd | grep -c "socket:"
ss -tanp | grep <pid> | awk '{print $1}' | sort | uniq -c

If you see thousands in CLOSE_WAIT, your code is not closing sockets the peer already closed. CLOSE_WAIT means the remote sent FIN and your side has not called close. The kernel cannot reclaim it for you.

In Node this is usually an HTTP client without an agent, or an agent with keepAlive on and no maxSockets bound:

const agent = new http.Agent({
  keepAlive: true,
  maxSockets: 50,
  maxFreeSockets: 10,
  timeout: 30000,
});

Without a timeout, a connection to a peer that has silently gone away holds its descriptor until the process exits. This is the same failure shape as timeouts you never set on outbound requests.

2. Streams opened per request and never destroyed

// leaks a descriptor on every error path
const stream = fs.createReadStream(path);
stream.pipe(res);

// closes it, including when the client disconnects mid stream
const stream = fs.createReadStream(path);
stream.pipe(res);
res.on('close', () => stream.destroy());

The error path is the one that leaks. Happy path code closes cleanly, then a client disconnects halfway through a download and the stream is orphaned.

3. Watchers

fs.watch, chokidar, and anything that recurses a directory tree consume one descriptor per watched directory on Linux via inotify. A node_modules tree will exhaust fs.inotify.max_user_watches long before it exhausts your file limit, and the error you get is ENOSPC, which is thoroughly misleading:

sysctl fs.inotify.max_user_watches
sysctl -w fs.inotify.max_user_watches=524288

ENOSPC from a file watcher does not mean the disk is full. It means you ran out of watches.

4. Connection pools sized per worker

A pool of 20 connections looks modest until you run 16 workers, at which point it is 320 connections and your database rejects the rest. I wrote about the database side of this in too many connections; the descriptor exhaustion is the same arithmetic viewed from the client.

Pool size is per process, not per fleet. Multiply by worker count before you decide it is reasonable.

Finding the leak rather than guessing

Sample the count over time. A leak is linear, a spike is load:

while true; do
  echo "$(date +%s) $(ls /proc/<pid>/fd | wc -l)"
  sleep 10
done

If the number climbs and never comes back down after traffic subsides, it is a leak. If it rises and falls with request volume, your limit is genuinely too low and raising it is the correct fix.

For the actual call site, lsof grouped by target tells you what is accumulating:

lsof -p <pid> | awk '{print $5, $9}' | sort | uniq -c | sort -rn | head -20

Prevention

• Set LimitNOFILE explicitly in the unit file rather than relying on distribution defaults, and verify it in /proc/<pid>/limits after deploy.
• Give every HTTP agent and database pool an explicit timeout and an explicit maximum.
• Alert on descriptor count as a fraction of the limit, not on the crash. Eighty percent sustained is a leak you have not noticed yet, which is exactly the kind of absence of signal that turns into an incident at the worst moment.
• Destroy streams on close, not only on finish.
• In containers, remember the limit comes from the runtime rather than the host shell. docker run --ulimit nofile=65535:65535 sets it for that container.