Elixir Development with Vim

I love Vim (more specifically, the NeoVim fork). Modal editing as part of my Unix IDE brings an immense amount of productivity and enjoyment to my day-to-day development activities. As such, whenever I take up a new language or framework, I enjoy experimenting with how best to integrate it into my existing workflow.

Having started programming in Elixir, I've started the customization journey for the language. Much as with Maslow's Hierarchy of Needs, there's a hierarchy of editor support required for an fulfilling programming experience. Let's get there with Elixir!

Read more…

Kicking the Tires with Elixir, Part 1 - Pipes

Lately I've been playing with Elixir, a functional language that sits over Erlang and the OTP framework. It's exciting because it handles embarrassingly scalable problems with aplomb, enabling a high level of parallelism and concurrency with a great developer experience. While I think Rust shines at the system programming level, Elixir seems like a perfect candidate for web services - balancing power with ergonomics.

There are plenty of well-written posts about the language and its associated libraries. I wanted to touch on what stood out to me as a Pythonista, web developer, and nerd. I don't know how many posts will comprise the series, but I have at least a few topics in mind.

First up: pipes.

Read more…

Migrating from Gmail to Fastmail

In 2004, Google launched Gmail. This service changed everything. You didn't have to worry about running over your few megabytes of quota - your storage space was "unlimited" (with a ticker and everything)! Deletion was a thing of the past, you now archive! Folders were so ninety-ninety-late, there were labels! I got an invite within the first two weeks of it launching, and it was good.

A decade or so later, Google launched Inbox, which brought innovations like bundles, snoozing, highlights, pinning, sweeping, and smart filtering to the deluge of email that flooded your account each day. I switched from Gmail to Inbox, and it was better.

Then, in true Google fashion, it was sacrificed at the altar of project mismanagement (or whatever the lack of product strategy is called). And it was bad.

Since being forced back to Gmail, I've constantly lamented the death of a service that made dealing with email less painful for me. Gmail is not only without new innovation, but it's also slow; It regularly fails to load new messages, or seemingly loses track of what it should be showing, necessitating a hard refresh. Sure, Google has thrown a few bones at it, like smart replies, but I respond to so few emails that spending a few seconds to formulate a response has never been an issue.

Given these concerns, I realized that the "stickiness" of Gmail was gone. I have the means to pay for service, and nothing is keeping me on Gmail (other than the fact that everyone's been using my Gmail address for 16 years). Leaving Gmail sounded doable, and I owned a personal domain on that I'd love to use for email. The only question was: where do I go for hosting?

Read more…

Optimizing Rust Binary Size

I develop and maintain a git extension called git-req. It enables developers to check out pull requests from GitHub and GitLab by their number instead of branch name. It initially started out as a bash script that invoked Python for harder tasks (e.g., JSON parsing). It worked well enough, but I wanted to add functionality that would have been painful to implement in bash. Additionally, one of my goals was to make it as portable as possible, and requiring a Python distribution be available flew against that. That meant that I needed to distribute this as a binary instead of a script, so I set about finding a programming language to use. After surveying what was available, and determining what would be the best addition to my toolbox, I selected Rust.

The programming language has a steep learning curve, but has been fun to learn and immerse myself within. The community is great, and I'm excited to find more opportunities to use Rust in the future.

The rewrite took a while to accomplish, but when all was said and done, everything worked, and worked well. I was able to implement some snazzy new features as well as polish some rough edges. However, for how "simple" I felt the underlying program to be, it clocked in at 13 megabytes. That felt like a lot. So, I decided to see what could be done.

Read more…

Elasticsearch Frustration: The Curious Query

Last year I was poking at an Elasticsearch cluster to review the indexed data and verify that things were healthy. It was all good until I stumbled upon this weird document:

{
  "_version": 1,
  "_index": "events",
  "_type": "event",
  "_id": "_query",
  "_score": 1,
  "_source": {
    "query": {
      "bool": {
        "must": [
          {
            "range": {
              "date_created": {
                "gte": "2016-01-01"
              }
            }
          }
        ]
      }
    }
  }
}

It may not be immediately obvious what's going on in the above snippet. Instead of a valid event document, there's a document with a query as the contents. Additionally, the document ID appears to be _query instead of the expected GUID. The combination of these two irregularities makes it seem as if someone accidentally posted a query to the wrong endpoint. No problem, just delete the document, right?

DELETE /events/event/_query
ActionRequestValidationException[Validation Failed: 1: source is missing;]

Wat.

Read more…

A DevOps Workflow, Part 3: Deployment

This series is a longform version of an internal talk I gave at a former company. It wasn't recorded. It has been mirrored here for posterity.

Congratulations, your code looks good! Now all you need to do is put your application in front of your users to discover all the creative ways they'll break it. In order to do this, we'll have to create our instances, configure them, and deploy our code.

Read more…

A DevOps Workflow, Part 2: Continuous Integration

This series is a longform version of an internal talk I gave at a former company. It wasn't recorded. It has been mirrored here for posterity.

Look at you – all fancy with your consistent and easily-managed development environment. However, that's only half of the local development puzzle. Sure, now developers can no longer use "it works on my machine" as an excuse, but all that means is they know that something runs. Without validation, your artisanal ramen may be indistinguishable from burned spaghetti. This is where unit testing and continuous integration really prove their worth.

Read more…

A DevOps Workflow, Part 1: Local Development

This series is a longform version of an internal talk I gave at a former company. It wasn't recorded. It has been mirrored here for posterity.

How many times have you heard: "That's weird - it works on my machine?"

How often has a new employee's first task turned into a days-long effort, roping in several developers and revealing a surprising number of undocumented requirements, broken links and nondeterministic operations?

How often has a release gone south due to stovepiped knowledge, missing dependencies, and poor documentation?

In my experience, if you put a dollar into a swear jar whenever one of the above happened, plenty of people would be retiring early to spend time on their private islands. The fact that this situation exists is a huge problem.

What would an ideal solution look like? It should ensure consistency of environments, capture external dependencies, manage configuration, be self-documenting, allow for rapid iteration, and be as automated as possible. These features - the intersection of development and operations - make up the practice of DevOps. The solution shouldn't suck for your team - you need to maximize buy-in, and that can't be done when people need to fight container daemons and provisioning scripts every time they rebase to master.

In this series, I'll be walking through how we do DevOps at HumanGeo. Our strategy consists of three phases - local development, continuous integration, and deployment.

Please note that, while I mention specific technologies, I'm not stating that this is The One True Way™. We encourage our teams to experiment with new tools and methods, so this series presents a model that several teams have implemented with success, not official developer guidelines.

Read more…

Switching to NeoVim (Part 2)

2016-11-03 Update: Now using the XDG-compliant configuration location.

Now that my initial NeoVim configuration is in place, I'm ready to get to work, right? Well, almost. In my excitement to make the leap from one editor to another, I neglected a portion of my attempt to keep Vim and NeoVim isolated - the local runtimepath (usually ~/.vim).

"But Aru, if NeoVim is basically Vim, shouldn't they be able to share the directory?" Usually, yes. But I anticipate, as I start to experiment with some of the new features and functionality of NeoVim, I might add plugins that I want to keep isolated from my Vim instance.

I'd like Vim to use ~/.vim and NeoVim to use ~/.config/nvim. Accomplishing this is simple - I must first detect whether or not I'm running NeoVim and base the root path on the outcome of that test:

if has('nvim')
    let s:editor_root=expand("~/.config/nvim")
else
    let s:editor_root=expand("~/.vim")
endif

With the root directory in a variable named editor_root, all that's left is a straightforward find and replace to convert all rtp references to the new syntax.

e.g. let &rtp = &rtp . ',.vim/bundle/vundle/'let &rtp = &rtp . ',' . s:editor_root . '/bundle/vundle/'

With those replacements out of the way, things almost worked. Almost.

I use Vundle. I think it's pretty rad. My vimrc file is configured to automatically install it and download all of the defined plugins in the event of a fresh installation. The first time I launched NeoVim with the above changes didn't result in a fresh install - it was still reading the ~/.vim directory's plugins.

Perplexed, I dove into the Vundle code. Sure enough, it appears to default to installing plugins to $HOME/.vim if a directory isn't passed in to the script initialization function. It appears that I was reliant on this default behavior. Thankfully, this was easily solved by passing in my own bundle path:

call vundle#rc(s:editor_root . '/bundle')

And with that, my Vim and NeoVim instances were fully isolated.

Switching to NeoVim (Part 1)

2016-11-03 Update: Now using the XDG-compliant configuration location.

NeoVim is all the rage these days, and I can't help but be similarly enthused. Unlike other editors, which have varying degrees of crappiness with their Vim emulation, NeoVim is Vim.

If it's Vim, why bother switching? Much like all squares are rectangles, but not all rectangles are squares, NeoVim has a different set of aspirations and features. While vanilla Vim has the (noble and important) goal of supporting all possible platforms, that legacy has seemingly held it back from eliminating warts and adding new features. That's both a good thing and a bad thing. Good because it's stable, bad because it can lead to stagnation. A Vim contributor, annoyed with how the project was seemingly hamstrung by this legacy (with its accompanying byzantine code structure, project structure, and conventions), decided to take matters into his hands and fork the editor.

The name of the fork? NeoVim.

It brings quite a lot to the table, and deserves a blog post or two in its own right. I'll leave the diffing as an exercise to the reader. I plan on writing about some of those differences as I do more with the fork's unique features.

Read more…