Ioerror: [errno 22] Invalid Argument When Reading/writing Large Bytestring

I'm getting IOError: [Errno 22] Invalid argument when I try to write a large bytestring to disk with f.write(), where f was opened with mode wb. I've seen lots of people online ge

Solution 1:

This appears to be a general OSX bug with fread / fwrite and so isn't really fixable by a Python user. See numpy #3858, this torch7 commit, this SO question/answer, ....

Supposedly it's been fixed in Mavericks, but I'm still seeing the issue.

Python 2 may have worked around this or its io module may have always buffered large reads/writes; I haven't investigated thoroughly.

Solution 2:

Perhaps try not opening with the b flag, I didn't think that was supported on all OS / filesystems.