March 04, 2011

Mark Twain on Whiskey and Civilization

I found this gem of a snippet today, and I rather agree with what it says!

“How solemn and beautiful is the thought that the earliest pioneer of civilization, the van-leader of civilization, is never the steamboat, never the railroad, never the newspaper, never the Sabbath-school, never the missionary — but always whiskey! Such is the case. Look history over; you will see.

The missionary comes after the whiskey — I mean he arrives after the whiskey has arrived; next comes the poor immigrant, with ax and hoe and rifle; next, the trader; next, the miscellaneous rush; next, the gambler, the desperado, the highwayman, and all their kindred in sin of both sexes; and next, the smart chap who has bought up an old grant that covers all the land; this brings the lawyer tribe; the vigilance committee brings the undertaker. All these interests bring the newspaper; the newspaper starts up politics and a railroad; all hands turn to and build a church and a jail — and behold! civilization is established forever in the land.

But whiskey, you see, was the van-leader in this beneficent work. It always is. It was like a foreigner — and excusable in a foreigner — to be ignorant of this great truth, and wander off into astronomy to borrow a symbol. But if he had been conversant with the facts, he would have said: Westward the Jug of Empire takes its way. ”

--

If you have any questions or thoughts, don't hesitate to reach out. You can find me as @viksit on Twitter.

December 08, 2010

Reading Java properties file in Clojure

A simple and effective way to read properties files in Clojure, since they transform into Clojure maps!

(into {} (doto (java.util.Properties.) (.load (-> (Thread/currentThread) (.getContextClassLoader) (.getResourceAsStream "log4j.properties")))))

Next, to actually read this in, using atoms to swap the values like this seems to work,

(def *args* (atom {:a 10, :b 20})) (defn -main (swap! *args* assoc :a (read) :b (read)))

--

If you have any questions or thoughts, don't hesitate to reach out. You can find me as @viksit on Twitter.

July 08, 2010

Decoding the US Military's Cyber Command Logo code

From this Wired article (http://www.fastsum.com/support/md5-checksum-utility-faq/md5-hash.php), it looks like there’s a number that is part of the cyber command’s logo – 9ec4c12949a4f31474f299058ce2b22a. Well, its 32 characters long, and looks like a hash. Sure enough, a quick python check later of the organization’s mission statement with md5 results in,

import hashlib >>> hashlib.md5("USCYBERCOM plans, coordinates, integrates, synchronizes and conducts activities to: direct the operations and defense of specified Department of Defense information networks and; prepare to, and when directed, conduct full spectrum military cyberspace operations in order to enable actions in all domains, ensure US/Allied freedom of action in cyberspace and deny the same to our adversaries.").hexdigest() '9ec4c12949a4f31474f299058ce2b22a'

Voila!

--

If you have any questions or thoughts, don't hesitate to reach out. You can find me as @viksit on Twitter.

June 20, 2010

Mutable vs Immutable datastructures - Serialization vs Performance

In my last post, I was playing around with methods to serialize Clojure data structures, especially a complex record that contains a number of other records and refs. Chas Emerick and others mentioned in the comments there, that putting a ref inside a record is probably a bad idea – and I agree in principle. But this brings me to a dilemma.

Lets assume I have a complex record that contains a number of "sub" records that need to be modified during a program's execution time. One scenario this could happen in is a record called "Table", that contains a "Row" which is updated (Think database tables and rows). Now this can be implemented in two ways,

  • Mutable data structures – In this case, I would put each row inside a table as a ref, and when the need to update happens, just fine the row ID and use a dosync – alter to do any modifications needed.
    • The advantage is that all data is being written to in place, and would be rather efficient.
    • The disadvantage however, is that when serializing such a record full of refs, I would have to build a function that would traverse the entire data structure and then serialize each ref by dereferencing it and then writing to a file. Similarly, I'd have to reconstruct the data structure when de-serializing from a file.
{:filename "tab1name", :tuples #<ref :field="" :tupdesc="" nil="">}, :tup #<ref :name="">} {:recordid nil, :tupdesc {:x #<ref :field="">}, :tup #<ref :name="">}}>, :tupledesc {:x #<ref :field="">}} </ref></ref></ref></ref></ref>
  • Immutable data structures – This case involves putting a ref around the entire table data structure, implying that all data within the table would remain immutable. In order to update any row within the table, any function would return a new copy of the table data structure with the only change being the modification. This could then overwrite the existing in-memory data structure, and then be propagated to the disk as and when changes are committed.
    • The advantage here is that having just one ref makes it very simple to serialize – simply de-ref the table, and then write the entire thing to a binary file.
    • The disadvantage here is that each row change would make it necessary to return a new "table", and writing just the "diff" of the data to disk would be hard to do.
#<ref :field="" :name="" :tup="" :tupdesc="" :tupledesc="" :tuples="" nil=""> <p> So at this point, which method would you recommend?</p> <p></p> </ref>

--

If you have any questions or thoughts, don't hesitate to reach out. You can find me as @viksit on Twitter.

June 19, 2010

Serializing Clojure Datastructures

I’ve been trying to figure out how best to serialize data structures in Clojure, and discovered a couple of methods to do so. (Main reference thanks to a thread on the Clojure Google Group (http://groups.google.com/group/clojure/browse_thread/thread/29a94dd74b8beaaa/a05b126b192195e9) )

(def box {:a 1 :b 2}) (defn serialize (with-open (.writeObject outp o))) (defn deserialize (with-open (.readObject inp))) (serialize box "/tmp/ob1.dat") (deserialize "/tmp/ob1.dat")

This works well for any Clojure data structure that is serializable. However, my objective is slightly more intricate – I’d like to serialize records that are actually refs. I see a few options for this,

– Either use a method that puts a record into a ref, rather than a ref into a record and then use the serializable, top level map
– Write my own serializer to print this to a file using clojure+read
– Use Java serialization functions directly.

Thoughts?

--

If you have any questions or thoughts, don't hesitate to reach out. You can find me as @viksit on Twitter.