Introduction
Repetitive duties are ripe for automation. It is not uncommon for builders and system directors to automate routine duties like well being checks and file backups with shell scripts. Nevertheless, as these duties develop into extra advanced, shell scripts might develop into more durable to take care of.
Luckily, we will use Python as an alternative of shell scripts for automation. Python supplies strategies to run shell instructions, giving us the identical performance of these shells scripts, with further help from Python’s ecosystem. Studying tips on how to run shell instructions in Python opens the door for us to automate laptop duties in a structured and scalable manner.
On this article, we are going to have a look at the assorted methods to execute shell instructions in Python, and the perfect scenario to make use of every technique.
Utilizing os.system to Run a Command
Python permits us to right away execute a shell command that is saved in a string utilizing the os.system() perform. Let’s begin by creating a brand new Python file referred to as echo_adelle.py and enter the next:
import os
os.system("echo Hey from the opposite facet!")
The very first thing we do in our Python file is import the os module, which incorporates the system perform that may execute shell instructions. The following line does precisely that, runs the echo command in our shell via Python.
In your Terminal, run this file with utilizing the next command, and it is best to see the corresponding output:
$ python3 echo_adelle.py
Hey from the opposite facet!
Because the echo instructions prints to our stdout, os.system() additionally shows the output on our stdout stream. Whereas not seen within the console, the os.system() command returns the exit code of the shell command. An exit code of 0 means it ran with none issues and some other quantity means an error. This manner, you ca management the stream of your instructions, verify the outcomes and both retry or return an error if any name fails:
import os
home_dir = os.system("cd ~")
print("`cd ~` ran with exit code %d" % home_dir)
unknown_dir = os.system("cd doesnotexist")
print("`cd doesnotexis` ran with exit code %d" % unknown_dir)
On this script, we create two variables that retailer the results of executing instructions that change the listing to the house folder, and to a folder that doesn’t exist. Operating this file, we are going to see:
$ python3 cd_return_codes.py
`cd ~` ran with exit code 0
sh: line 0: cd: doesnotexist: No such file or listing
`cd doesnotexist` ran with exit code 256
The primary command, which modifications the listing to the house listing, executes efficiently. Subsequently, os.system() returns its exit code, zero, which is saved in home_dir. Then again, unknown_dir shops the exit code of the failed bash command to alter the listing to a folder that doesn’t exist.
The os.system() perform executes a command, prints any output of the command to the console, and returns the exit code of the command. If we want extra advantageous grained management of a shell command’s enter and output in Python, we should always use the subprocess module.
Operating a Command with subprocess
The subprocess module is Python’s beneficial approach to executing shell instructions. It offers us the flexibleness to suppress the output of shell instructions or chain inputs and outputs of assorted instructions collectively, whereas nonetheless offering an analogous expertise to os.system() for fundamental use instances.
In a brand new file referred to as list_subprocess.py, let’s run a command with the subprocess module:
import subprocess
list_files = subprocess.run(["ls", "-l"])
print("The exit code was: %d" % list_files.returncode)
Within the first line, we import the subprocess module, which is a part of the Python customary library. We then use the subprocess.run() perform to execute the command. Like os.system(), the subprocess.run() command returns the exit code of what was executed.
In contrast to os.system(), word how subprocess.run() requires an inventory of strings as enter as an alternative of a single string. The primary merchandise of the listing is the identify of the command. The remaining gadgets of the listing are the flags and the arguments of the command.
Be aware: As a rule of thumb, it’s worthwhile to separate the arguments based mostly on house, for instance ls -alh could be ["ls", "-alh"], whereas ls -a -l -h, could be ["ls", "-a", -"l", "-h"]. As one other instance, echo good day world could be ["echo", "hello", "world"], whereas echo "good day world" or echo good day world could be ["echo", "hello world"].
Run this file and your console’s output could be much like:
$ python3 list_subprocess.py
whole 80
[email protected] 1 stackabuse workers 216 Dec 6 10:29 cd_return_codes.py
[email protected] 1 stackabuse workers 56 Dec 6 10:11 echo_adelle.py
[email protected] 1 stackabuse workers 116 Dec 6 11:20 list_subprocess.py
The exit code was: 0
Now let’s attempt to use one of many extra superior options of subprocess.run(), particularly ignore output to stdout. In the identical list_subprocess.py file, change:
list_files = subprocess.run(["ls", "-l"])
To this:
list_files = subprocess.run(["ls", "-l"], stdout=subprocess.DEVNULL)
The usual output of the command now pipes to the particular /dev/null system, which suggests the output wouldn’t seem on our consoles. Execute the file in your shell to see the next output:
$ python3 list_subprocess.py
The exit code was: 0
What if we needed to supply enter to a command? The subprocess.run() facilitates this by its enter argument:
Try our hands-on, sensible information to studying Git, with best-practices, industry-accepted requirements, and included cheat sheet. Cease Googling Git instructions and truly be taught it!
import subprocess
useless_cat_call = subprocess.run(["cat"],
stdout=subprocess.PIPE,
textual content=True,
enter="Hey from the opposite facet")
print(useless_cat_call.stdout)
We use subprocess.run() with fairly a couple of instructions, let’s undergo them:
stdout=subprocess.PIPEtells Python to redirect the output of the command to an object so it may be manually learn latertextual content=Truereturnsstdoutandstderras strings. The default return kind is bytes.enter="Hey from the opposite facet"tells Python so as to add the string as enter to thecatcommand.
Operating this produces the next output:
Hey from the opposite facet
We will additionally elevate an Exception with out manually checking the return worth. In a brand new file, false_subprocess.py, add the code under:
import subprocess
failed_command = subprocess.run(["false"], verify=True)
print("The exit code was: %d" % failed_command.returncode)
In your Terminal, run this file. You will notice the next error:
$ python3 false_subprocess.py
Traceback (most up-to-date name final):
File "false_subprocess.py", line 4, in <module>
failed_command = subprocess.run(["false"], verify=True)
File "/usr/native/python/3.7.5/Frameworks/Python.framework/Variations/3.7/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['false']' returned non-zero exit standing 1.
By utilizing verify=True, we inform Python to boost any exceptions if an error is encountered. Since we did encounter an error, the print assertion on the ultimate line was not executed.
The subprocess.run() perform offers us immense flexibility that os.system() does not when executing shell instructions. This perform is a simplified abstraction of the subprocess.Popen class, which supplies further performance we will discover.
Operating a Command with Popen
The subprocess.Popen class exposes extra choices to the developer when interacting with the shell. Nevertheless, we have to be extra express about retrieving outcomes and errors.
By default, subprocess.Popen doesn’t cease processing of a Python program if its command has not completed executing. In a brand new file referred to as list_popen.py, kind the next:
import subprocess
list_dir = subprocess.Popen(["ls", "-l"])
list_dir.wait()
This code is equal to that of list_subprocess.py. It runs a command utilizing subprocess.Popen, and waits for it to finish earlier than executing the remainder of the Python script.
For instance we don’t wish to look ahead to our shell command to finish execution so this system can work on different issues. How wouldn’t it know when the shell command has completed execution?
The ballot() technique returns the exit code if a command has completed working, or None if it is nonetheless executing. For instance, if we needed to verify if list_dir was full as an alternative of look ahead to it, we might have the next line of code:
list_dir.ballot()
To handle enter and output with subprocess.Popen, we have to use the talk() technique.
In a brand new file referred to as cat_popen.py, add the next code snippet:
import subprocess
useless_cat_call = subprocess.Popen(["cat"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, textual content=True)
output, errors = useless_cat_call.talk(enter="Hey from the opposite facet!")
useless_cat_call.wait()
print(output)
print(errors)
The talk() technique takes an enter argument that is used to move enter to the shell command. The talk technique additionally returns each the stdout and stderr when they’re set.
Having seen the core concepts behind subprocess.Popen, we’ve got now coated 3 ways to run shell instructions in Python. Let’s re-examine their traits so we’ll know which technique is greatest suited to a challenge’s necessities.
Greatest Method to Run Shell Instructions with Python?
If it’s worthwhile to run one or a couple of easy instructions and don’t thoughts if their output goes to the console, you should use the os.system() command. If you wish to handle the enter and output of a shell command, use subprocess.run(). If you wish to run a command and proceed doing different work whereas it is being executed, use subprocess.Popen.
Here’s a desk with some usability variations you could additionally use to tell your resolution:
| os.system | subprocess.run | subprocess.Popen | |
|---|---|---|---|
| Requires parsed arguments | no | sure | sure |
| Waits for the command | sure | sure | no |
| Communicates with stdin and stdout | no | sure | sure |
| Returns | return worth | object | object |
Conclusion
Python permits you to execute shell instructions, which you should use to start out different applications or higher handle shell scripts that you just use for automation. Relying on our use case, we will use os.system(), subprocess.run() or subprocess.Popen to run bash instructions.
Utilizing these strategies, what exterior process would you run through Python?
