Practices for programming in a scientific environment? [closed]

What languages/environments have you used for developing scientific software, esp. data analysis? What libraries? (E.g., what do you use for plotting?) I used to work for Enthought, the primary corporate sponsor of SciPy. We collaborated with scientists from the companies that contracted Enthought for custom software development. Python/SciPy seemed to be a comfortable environment for … Read more

Interactive large plot with ~20 million sample points and gigabytes of data

So your data isn’t that big, and the fact that you’re having trouble plotting it points to issues with the tools. Matplotlib has lots of options and the output is fine, but it’s a huge memory hog and it fundamentally assumes your data is small. But there are other options out there. So as an … Read more

binning data in python with scipy/numpy

It’s probably faster and easier to use numpy.digitize(): import numpy data = numpy.random.random(100) bins = numpy.linspace(0, 1, 10) digitized = numpy.digitize(data, bins) bin_means = [data[digitized == i].mean() for i in range(1, len(bins))] An alternative to this is to use numpy.histogram(): bin_means = (numpy.histogram(data, bins, weights=data)[0] / numpy.histogram(data, bins)[0]) Try for yourself which one is faster… … Read more