How To Enforce Only One Running Instance Of A Process In Python Django Framework?

I have a python Django manage command that should be called upon receiving an input file but this command is not safe for parallel calls. So an input file should be processed only

Solution 1:

As KlausD mentions in his comment, the canonical (and language-agnostic) solution is to use a lock file containing the pid of the running process, so the code responsible for the lock acquisition can check if the process is still running.

An alternative solution if you use redis in your project is to store the lock in redis with a TTL that's a bit longer than the worst case runtime of the task. This makes sure the lock will be freed whatever, and also allow to easily share the lock between multiple servers if needed.

EDIT:

is it possible that the process crashes and another process pick up the same pid?

Yes, of course, and that's even rather likely (and this is an understatement) on a server running for month or more without reboot, and even more so if the server runs a lot of short-lived processes. You will not only have to check if there's a running process matching this pid but also get the process stats to inspect the process start time, the command line, the parent etc and decides the likelyhood it's the same process or a new one.

Note that this is nothing new - most process monitoring tools face the same problem, so you may want to check how they solved it (gunicorn might be a good starting point here).