urn:jonoflayham-io:feed2015-03-08T00:00:00.000Z2015-03-08T00:00:00.000ZJon Woodsurn:jonoflayham-io:feed:post:Thank you<p>Thank you, <a href="https://github.com/gilbertw1/blog-gen">Bryan Gilbert</a>, <a href="https://github.com/cjohansen/cjohansen-no/blob/master/resources/md/building-static-sites-in-clojure-with-stasis.md">Christian Johansen</a> and other authors of the <a href="http://clojure.org">Clojure</a> code behind this new blog. I'm finding that understanding and using Clojure is a refreshing antidote to the current day job, which is dismayingly full of SOAP, Maven, Spring, things called Websphere.* and someone else's copy-and-paste Java. Never mind! Not for much longer, then I'm out into the warm sunlight of a new opportunity.</p>2016-03-28T00:00:00.000ZJon Woodsurn:jonoflayham-io:feed:post:Clojure for an experienced Java developer<p><em>I'd promised a friend at work that I would put together a few suggestions for learning Clojure, and here they are. He has masses of experience developing in other languages - mostly Java - and is a deep and quick thinker. We've already talked a bit about Clojure and he's seen some in action, but is otherwise new to the language and its ecosystem.</em></p><h1>Build tool</h1><p>For a build and dependency management tool, let's go with Leiningen for now, since it's pretty much the standard. One of the nice things about the Clojure ecosystem is that questioning the <em>status quo</em> is the norm, so don't worry that you'll become the equivalent of a Mavenite. </p><h1>Clojure version</h1><p>The latest stable Clojure is at version 1.8. You specify it just like any other project dependency, in Leiningen's <code>project.clj</code> file.</p><h1>Tools</h1><h2>Cursive</h2><p>You've got Cursive installed, and you have access to an ordinary Leiningen REPL (via <code>lein repl</code>). That should be enough. We've talked about Emacs, and I gather that for you as for me it wouldn't be helpful go down that road at the moment.</p><p>Cursive has 3 editing modes: plain, paredit, and parinfer. The last of these is new to the most recent version of Cursive. The mode's shown at the very bottom right of IntelliJ. I'd use paredit for minimum surprises and maximum usefulness.</p><p>You should probably get to grips fairly early on with the interplay between source code 'in' a project and what the REPL is evaluating, just so you can quickly get out into the clear waters of guided experimentation and even productivity. In the early days you'll inevitably end up being puzzled about why a change hasn't 'taken' or why you can't evaluate something. Get used to the Cursive keyboard shortcuts, or at least menu options, for 'load this file in REPL' and friends, and when you come to understanding namespaces more deeply (more below) you'll really benefit.</p><p>Off the top of my head, places to explore in IntelliJ re Cursive/Clojure: the Run menu; right-clicking on the top of the project structure; the Edit/Structural editing menu; Preferences/Keymap/Clojure Keybindings.</p><h2>REPLs in general</h2><p>A REPL is in some namespace. It has access to definitions you've evaluated in it, whether by typing them or by asking eg Cursive to evaluate some source code in the REPL context. It has history, so up-arrow works. For the Leiningen REPL, history is stored in the current working directory in <code>.lein-repl-history</code>; dunno about Cursive's history. REPLs also generally have auto-complete - start typing then use tab.</p><p>There are 3 or 4 really useful REPL functions. They're in the clojure.repl namespace. When invoking these functions you don't start off by having to mention the name space, but depending on what you do with namespaces in your REPL session you may find you later have to specify the namespace explicitly. So e.g.</p> <pre><code class="clj"> (doc interpose) (clojure.repl/doc interpose) (clojure.repl/source clojure.repl/source) </code></pre><p>The last prints its own source code.</p><p>Of course, since you're in IntelliJ you can delve into source code in the ways you'd expect (command-click, command-b etc).</p><h1>Going deep</h1><p>While you're learning Clojure and getting to grips with the fundamentals, I heartily support your natural tendency to go deep first. Understand why the REPL prints exactly what it does; think deeply about evaluation contexts and lifecycles; and keep track of those questions which you can't solve yourself - I'd be very happy to try to answer them with you. Since I'm not very far ahead of you, I'm sure I'd learn in the process...</p><p>There'll be two particular wrinkles to smooth over in this early phase.</p><h2>Early wrinkle 1: stack traces</h2><p>You'll probably be baffled along the way by stack traces. Both the Leiningen REPL and Cursive's will shorten exception reports. In both cases, <code>*e</code> evaluates to the last exception met. If you want grab hold of that to take a look, <code>(def bla *e)</code> then you've got <code>bla</code>, without worrying about <code>*e</code> being overwritten by your next experiment. Cursive will pretty-print a useful representation of exceptions when you evaluate them - you could try that out early on:</p> <pre><code class="clj"> wibble ; Cursive reports &quot;huh?&quot; as an exception. Save the last one: (def bla *e) ; Now evaluate the exception you&#39;ve captured, and take a look bla </code></pre><p>There are libraries for colorising stack traces, rolling them up etc etc, making them more expressive - but they can wait and are a matter of personal preference. The're just JVM exceptions, at the end of the day. Leave that kind of thing til you're trying to be fluent and uber-productive.</p><h2>Early wrinkle 2: namespaces</h2><p>Namespaces are hierarchical contexts in which symbols live.</p><p>For all that Clojure champions statelessness, where namespaces are concerned the evaluation context (of source code being compiled, of evaluations you request in a REPL) is very stateful indeed.</p><p>You are always 'in' some namespace, whether you're at some point in time in your use of a REPL or you are the Clojure reader going through the process of compiling a source code file. You can change the namespace you're in, at any point. Swapping in and out of namespaces is generally something you'd only do in REPL contexts - in source code files, you generally declare the namespace stuff at the top and that's that.</p><p>Before you can refer to symbols in a namespace, you have to load (read and evaluate and compile) a source containing those symbols defined in that namespace. The source generally has to be on the classpath, which is defined by your build tool (more often than not, Leiningen).</p><p>You can almost always refer to a symbol by fully qualifying it using its namespace, provided that you've loaded the corresponding source.</p><p>Multiple source files/locations can contribute symbols to the same namespace, or even to different namespaces - but the convention is generally one file = one namespace = all of the symbols in that namespace.</p><p>When you declare the namespace of a source file, you generally use the <code>ns</code> macro, and it's that you should focus on understanding. At any rate and however you do it, in source files you declare which namespace you're in and also say what other namespaces your source uses and how you'd like to refer to them. The choices you make here are yours, and are all about brevity/expressivity. If you use only one function from a foreign namespace, and use it only once, you might as well fully qualify it. If you use a foreign function a lot, you'll probably want to (use the ns macro or some other mechanism to) alias its namespace or do away with having to mention it at all... unless the function name clashes with another.</p><p>Namespaces are initially one of the most confusing things about Clojure, for several reasons:</p> <ul> <li>there are several ways of doing the same thing</li> <li><code>ns</code> is a macro which therefore doesn't need its arguments quoted, whereas namespace-related <code>require</code>, <code>use</code>, <code>alias</code> and friends are functions whose arguments are evaluated by the time they get to them</li> <li><code>ns</code> in particular, being a macro, is free to make use of idomatic Clojure in which keywords (<code>:require</code>, <code>:refer</code>, <code>:only</code>...) are interspersed with symbols, vectors etc in an initially baffling way</li> </ul><p>However, be reassured that everything is pretty logical, and definitely deterministic! I suggest you let yourself run into a few difficulties to build up a desire to grok the subject, then go and read <a href="http://www.braveclojure.com/organization/">http://www.braveclojure.com/organization/</a> to understand it all much more deeply. Warning: that goes into the underlying mechanism of namespace management, which you may never need to know about. If it all gets too confusing, leave it for a while and come back to it after a bit more real-world experience.</p><p>By the way, since the namespace mechanism is by and large completely accessible to real Clojure, there's a healthy tendency for people to build tools which augment/improve/subvert the whole structure. In other contexts this might be worrying ("Oh no! Developers have power! Things are bound to go wrong!") but Clojurists don't seem to be so afraid of thinking.</p><h1>A good teaching narrative: Clojure for the Brave and True</h1><p>If you'd like something with a nice narrative and crystal clear explanations (and only the occasional piece of style which may not suit everybody) then I'd recommend <a href="http://www.braveclojure.com">http://www.braveclojure.com</a>. You have the book on your desk at work, but the online version is great - and even styled with a nice light vellum background! This will get you a long way - beyond me atm :) Personally I'd ignore its advice to use Emacs unless that really grabs you.</p><h1>A book for inspiration and reassurance: Clojure Cookbook</h1><p>Once the number of wtf?! moments is down to a manageable level when you look at Clojure code, the best book I can recommend is the Clojure Cookbook. Real-world problems, using real-world libraries, with succinct explanations and things you can easily try out. Core things (joining two lists, mapping functions over things), cool things (eg core/async) plus the real messy stuff (using Lucene, stashing things in Redis, generating PDFs...). The only downside is that it was published in 2014, but I should think all the libraries it mentions, and certainly all the patterns, are relevant.</p><p>It's available as a book/an ebook from O'Reilly via <a href="http://clojure-cookbook.com/">http://clojure-cookbook.com/</a> or Amazon etc, or in source code at <a href="https://github.com/clojure-cookbook/clojure-cookbook">https://github.com/clojure-cookbook/clojure-cookbook</a>.</p><h1>The community</h1><p>Follow <a href="https://twitter.com/hashtag/clojure">#clojure</a> to a while to get a flavour of what's going on and how people in the community interact. More at <a href="http://clojure.org/community/resources">http://clojure.org/community/resources</a>.</p>2017-02-04T00:00:00.000ZJon Woodsurn:jonoflayham-io:feed:post:Make your own tools<p>As developers, we're far too fond of the blunt, primitive tools we find lying around at the bottom of the software development stack - logs, CI GUIs, low-level bash commands, inflexible build scripts. Too rarely do we write bespoke, higher-order tools to make ourselves more productive and the job more enjoyable. We're missing out on real opportunities to go faster.</p><p>When we're writing production or test code which gets repetitive or unwieldy, it's not long before we naturally break it up into chunks and compose those chunks together at a higher, more useful level, then as things grow still more we refactor <em>that</em> and compose over it, and so on up the ladder of abstraction. We don't end up writing files of source code thousands of lines long, or at least I hope not. But when it comes to the <em>machinery</em> of software development, we're perfectly happy bolting together the same old steps by hand. We seem unaware that we're death-marching through numerous repetitive, un-factored-out, long-winded, ambiguously-defined, error-prone steps time and time again.</p><p>What would be so wrong in daring to create project-specific tools? Not just traditional automation, but an interface which is deliberately, intimately tailored to the software we're currently engaged in producing? It could be somethign we use at the command line or REPL, or even through its own GUI. </p><p>There are many reasons why we tend not to write our own tools, or tend not to give much love to the ones we do create.</p> <ol> <li><p>We can get buy using the facilities we already have, albeit inefficiently, so we allow ourselves to be drugged by comforting familiarity and forget that it's possible to improve.</p></li> <li><p>When we're not very experienced, we take it as read that everything's been done for us. "Surely Maven is all you need!"</p></li> <li><p>It's quite easy to write a wiki page to capture complex, often-repeated processes. You get to be a published author! Your effort is evident to others, and they'll thank you for it. And it's comforting to follow instructions by rote - at least for the first few times.</p></li> <li><p>We're easily demotivated by the effort required to make something new, even if its utility is blindingly obvious.</p></li> <li><p>Admittedly, automation sometimes hides things which it's useful to see, especially at the start of a project or a new kind of activity. That's why manual testing is still valuable. But this shouldn't stop us continually assessing how to avoid day-to-day drudgery.</p></li> <li><p>When we're working at scales where the effort definitely would be worth our while, there's often too much weight of opinion behind old practices, and we're engaged in so much fire-fighting - ironically often exacerbated by the very lack of tooling - that we can't think clearly anyway.</p></li> <li><p>Software development work is often ultimately dictated or influenced by someone who hasn't experienced the value of taking stock and investing in improving the delivery process itself. "Delivery-focussed" people like this see a team writing its own tools as a self-indulgence.</p></li> <li><p>Writing tools to solve messy software development problems often involves understanding many kinds of technologies - eg in infrastructure or cloud automation - and not everyone has the confidence nor the breadth of knowledge required to bring them all together.</p></li> <li><p>There might be a Product out there which Already Does This. Writing something specific to your situation might be seen as unnecessary effort, even if there's every chance the enterprise equivalent is expensive, bloated and doesn't in practice do what you need, or at least not without considerable customisation. And money. And formal procedures for requesting installation. And teams of experts. And time spent finding nothing useful about it on StackOverflow. And restrictions born of the fact that it's shared by many people. And downtime.</p></li> <li><p>Larger companies, especially, tend to mandate the use of specific toolsets, and explicitly making more tools is officially naughty. "We already have everything we need!" Which will either be nothing or one of those bloated enterprise things.</p></li> <li><p>Ground-up efforts to improve software delivery often see developers use tools, languages or patterns which are novel to the rest of the team, and as a result the initiative is frowned upon.</p></li> <li><p>Conversely, when it comes to writing tools we're often suckered by convention into believing we can't use the first-class programming language we're using in our core deliverables.</p></li> <li><p>When we're done writing our tools, there might not be anywhere acceptable to host or distribute whatever we've written, and so it won't get used and can't be promoted.</p></li> <li><p>Finally, some of us give the whole 'invent what you need' principle a bad name because in the past we've applied it for the wrong reasons, preferring to solve an easier, more interesting and mostly unrelated problem rather than the one our clients care about and are paying us for.</p></li> </ol><p>.....</p><p>Once you start thinking like this, the ideas pile on thick and fast. In a way, that's another reason why I've failed in the past to take the first step, because by the time I steel myself up to it, the first step looks decidedly drab compared to the grandiose vision in my head, and it all seems unachievable. "Time travel! Fantastic! Oh, wait - what shall I use for the UI?" Once I realise my first step can't get me straightaway to something like <a href="http://worrydream.com/">Brett Victor</a>'s <a href="https://vimeo.com/36579366">round-tripping</a> <a href="https://www.youtube.com/watch?v=PUv66718DII">code editor</a>, I give up then and there.</p><p>So let's come back down to reality. The first few steps to giving ourselves a tool like the one I've described ought to be really simple. If we give ourselves permission we'll see more and more opportunities for improvement using the skills we already have at our fingertips. And that is the most motivating thing about delivering software: making a useful difference more quickly than last time, getting to powerful and interesting levels of abstraction, and giving yourself more time to think.</p>