The problem here is that SHA256Managed reads 4096 bytes at a time (inherit from FileStream and override Read(byte[], int, int) to see how much it reads from the filestream), which is too small a buffer for disk IO. To speed things up (2 minutes for hashing 2 Gb file on my machine with SHA256, 1 … Read more
This is the fastest version I have found so far, about 6 times faster than readLines. On a 150MB log file this takes 0.35 seconds, versus 2.40 seconds when using readLines(). Just for fun, linux’ wc -l command takes 0.15 seconds. public static int countLinesOld(String filename) throws IOException { InputStream is = new BufferedInputStream(new FileInputStream(filename)); … Read more
How about making everything easier with Json.NET? public void LoadJson() { using (StreamReader r = new StreamReader(“file.json”)) { string json = r.ReadToEnd(); List<Item> items = JsonConvert.DeserializeObject<List<Item>>(json); } } public class Item { public int millis; public string stamp; public DateTime datetime; public string light; public float temp; public float vcc; } You can even get … Read more
I discovered git-annex recently which I find awesome. It was designed for managing large files efficiently. I use it for my photo/music (etc.) collections. The development of git-annex is very active. The content of the files can be removed from the Git repository, only the tree hierarchy is tracked by Git (through symlinks). However, to … Read more
Free read-only viewers: Large Text File Viewer (Windows) – Fully customizable theming (colors, fonts, word wrap, tab size). Supports horizontal and vertical split view. Also support file following and regex search. Very fast, simple, and has small executable size. klogg (Windows, macOS, Linux) – A maintained fork of glogg. Its main feature is regular expression … Read more