Python Trace Module - Trace Lines As They Are Executed, But Save To File, Rather Than Stdout

I want to trace the lines of a python script as they are executed. However the programme I use needs to print things to stdout. The trace option to the python trace module prints t

Solution 1:

You could copy the trace module code, and make a few changes to get it to write its output to a file of your choosing. There are five print statements between lines 600 and 650 that are the ones you want to change. Since you don't need to make it too pretty, you can add this to the very end of the file:

my_out_file = open("/home/mytrace.txt", "w")

and change the print statements so this:

print"%s(%d): %s" % (bname, lineno,
                      linecache.getline(filename, lineno)),

becomes this:

print >>my_out_file, "%s(%d): %s" % (bname, lineno,
                      linecache.getline(filename, lineno)),

Solution 2:

As per the docs of trace module, the outfile is only used to write updated count information.

• http://docs.python.org/library/trace.html

Trace module will continue to print the trace information to stdout.

For a manual tracing, if there is an entry function, I have used sys.settrace to do the work of tracing call flows. It can be extended to trace line by line execution. You can always file the information to a file rather than print it to stdout.

• http://docs.python.org/library/sys.html

A simple structure is

import sys
import os
import linecache

trace_depth = 0deftrace(f):

    defglobaltrace(frame, why, arg):
        global trace_depth
        if why == "call":
            # function call event , extract information
            .....
            pass

            trace_depth = trace_depth + 1returnNonedeflocaltrace(frame, why, arg):
        global trace_depth
        if why == "line":
            # line execution eventpasselif why == "return":
            trace_depth = trace_depth - 1# function return eventreturn localtrace

    def_f(*args, **kwds):
        sys.settrace(globaltrace)
        result = f(*args, **kwds)
        sys.settrace(None)
        return result

    return _f

globaltrace and localtrace are callback functions that are called with the event - "why"

say for event when a function is called, you can extract the information from the frame details.

if why == "call":
    # Parent frame details
    p_func = frame.f_back.f_code.co_name
    p_file = frame.f_back.f_code.co_filename
    p_lineinfo = frame.f_back.f_lineno

I have posted the full details here.

Solution 3:

It may happen that you need so-called 'function trace' described for PHP. I created Python tool that produces similar output using mentioned by pyfunc sys.settrace() hook.

Solution 4:

Are you invoking trace at the command line? If so, I recommend using the standard method of rerouting any stdout to any file. Redirect all output to file

foo > allout.txt 2>&1

I'm sure windows has something similar if you're in windows.

Solution 5:

Another possibility when you're using stdout/stderr for something else is to start a thread to write and feed messages to a queue

import threading
from queue import Queue

defworker(Q, file_dest):
    withopen(file_dest, "r+", buffering=1) as fh:
        whileTrue:
            fh.write(f"{}\n".format(Q.get()))

...
    dest_file = "some-writeable/path.txt"
    Q = Queue()  # can be a global, but please name it better!
    t = threading.Thread(
        worker,
        args=(Q, dest_file),
        daemon=True,  # don't block exit
    )
    # set t.daemon = True here instead in earlier Python
    t.start()

whenever you want to write, use the .put() method

Q.put("some message")

buffering=1 is used for line-buffering