Joao Moura

Engineering Leadership: Balancing Technical and Managerial Responsibilities

Tue, 11 Apr 2023 00:01:00 +0000

Ah, the age-old conundrum: as an engineering leader, how do you strike the perfect balance between your technical and managerial responsibilities? Fear not, for you’re not alone in this struggle!

The Great Shift and The Managerial Mindset

As you move up the ranks in your engineering career, it’s natural to find yourself with fewer hands-on technical tasks and more managerial duties. It’s all part of the journey, so don’t fret! But how do you navigate this transition without feeling like you’re losing your technical edge?

First, understand that this balancing act is something that all managers grapple with. Many managers question if they’re becoming technically stale, but rest assured, it’s normal to feel this way. Remember that your performance as a manager is measured by the aggregate performance of your team. It’s less about you and more about enabling others. Your main goal is to create an environment where your team can thrive and excel.

That said your ability to take informed decisions depends on having technical context (what is different from technical knowledge itself), so even though you primary responsability is not technical work, there is a lot of value on acquiring technical context, especially domain-specific knowledge about the code paths your team is responsible for.

As a manager, knowing every technical detail is neither realistic nor scalable. Instead, focus on understanding the broader technical landscape, and trust your team to handle the nitty-gritty.

Knowledge vs. Experience and Staying Up-to-Date

There’s a difference between technical knowledge and technical experience. As you progress in your managerial career, prioritize maintaining your technical experience over being an encyclopedia of technical knowledge. Staying up-to-date with technical changes is valuable and will help you with foward-thinking, but make sure you do so without becoming a blocker for your team or dropping the ball on other fronts. A good rule of thumb I’ve use is asking myself “is this the most impactful thing I could be doing to help my team achieve our goals”

Asking the Right Questions

Ultimately, good management is less about making the right technical decisions and more about asking the right questions. This is where your technical experience comes into play. By using your experience to guide your team, you’ll foster an environment where innovation and success can flourish.

So, there you have it! The key to balancing technical and managerial responsibilities lies in understanding your role, focusing on the big picture, and leveraging your technical experience to ask the right questions. Embrace the challenge and watch your team soar! 🚀

For people transitioning from Programming to Management

Sun, 03 Jun 2018 00:01:00 +0000

Management may sound boring for most developers, but as we go through our career it becomes clear that it’s a key role for a company. A good manager can make a huge difference in building great teams and awesome products.

Some developers want to build new stuff and have a deep interest on the technical side of things. Others might show a great set of soft skills and care for their teammates, for those, the jump to management could be a good option.

These are just some of the traits of a good manager, but when doing the jump there is much more that should be taken into account. The struggle involved in the jump is pretty real and that’s what I want to discuss in this article.

Most developers struggle when moving to a managerial role, I am part of this group. But once you understand why you are struggling you can finally start to work on tools to get better at it. As I got better as a manager I started to get emails, asking me for help, and that’s what lead me to write this article, provide an easy way for future managers to understand their struggles and better deal with it.

There is a bunch of reasons that make the transition to management so hard, the initial months can be a real struggle for those without experience. Understanding the issues is the initial step to handle with it, so let’s talk about some of the reasons why it feels so hard to make the jump.

Feedback Loop

Developers are used to have short feedback loops, think about it: you can start a new task by chatting with your teammates, then you can do some pair programming, and later you’ll have a code review cycle. That is not all, depending on your team’s process you’ll have retrospectives and daily meetings.

When transitioning to a managerial position most of that goes away. You can still ask for help, and I actually encourage you to do it, but there will be no “code reviews” nor “pair programming” on your decisions. You won’t be able to know if you are doing a good job until you let some of your decisions mature and start showing results.

You can still ask for feedback and I recommend you to do it as well, if you provide the correct environment, you can get an awesome result from that. But to see the results of your actions you will need to give it some time, good results take time.

Not having external validation can be a huge issue for most people, it can easily undermine one’s confidence, that’s why you need to be able to fight back.

One tool I used to deal with it was having quick review notes, I wrote down the decisions I took and weeks later I’d review that, asking myself if I would take the same decision once again.

Another good resource is to have other managers that you can share experiences with, in a way that you can support each other. Don’t be afraid of asking for help, most of the managers I talked with so far are always eager to help a fellow manager.

Fast Context Switching

As developers we are used have to laser focus into tasks for long hours, context switching is something that we are not used to, it’s always a pain to stop a task to attend a meeting or something else.

The problem is that when managing you must master context switching, it’s actually a great part of the job. Being able to go from one meeting to another, to talk about different tasks, while paying attention to it all, making sure everyone has what it needs.

This requires practice, it sucks to feel lost between different topics but you must power through it. One thing that helped me was to keep notes, physical notes from topics I need to get back and dig further. I visit this notebook every day, many times a day, mostly before meetings, making sure I got everything I needed covered. Writing things down allowed me to context-switch without getting worried about forgetting things.

Delegating

This is hard, the good thing is that if you ever was a team/technical leader you probably have some experience with this already. For those that never had to delegate tasks this can be a huge issue to overcome. Not doing it properly can result in micromanaging and that won’t benefit anyone, you will be stressed trying to keep up with everything and your team will feel like you don’t trust them. That will also prevent them from learning from their own mistakes, and from growing.

You need to take the jump. Accept that sharing responsibilities will make it easier for you to do a better job and also help your team to improve their skills.

Recovering

Managing can be stressful, it drains physical and mental energy. It also drains sentimental energy, yup you read it right, being responsible for other people takes a huge load on mental health, anxiety and stress can easily kick in if you don’t watch yourself.

Spare time to recovery, smile, have fun, make sure to invest in healthy relationships. These simple things can have a huge impact on your work.

I tried to keep this post short, but if you’re interested in knowing more about management in general make sure to let me know over twitter. I’m always eager to help other and share experiences.

Making Code Reviews Better

Sun, 03 Jun 2018 00:01:00 +0000

How many of you are doing code reviews every day? Do you enjoy it? Is it fun? Or do you do it because you feel like you have to? I know that is a lot of questions to start an article, but I want to make a point here, so bear with me just for one more question.

Why do we do PR reviews?

Most developers would say that we do reviews to catch bugs, but that is not it. Code review can actually be about way more than this. On pull requests, we are only seeing a part of it, essentially just a git diff , so you can catch syntax errors, but we can’t see how that interacts with the rest of the system just yet, so if reviewing a PR is not about catching errors then what is it about?

After a research conducted within real companies about pull requests, the three main benefits an engineering team can get out of reviewing PRs are:

Knowledge Transfer
Increased Team Awareness
Finding Alternative Solutions

Code Review is the discipline of explaining and discussing your code with your peers and that goes a long way to becoming a better software engineer. It’s actually a communication resource that gives the opportunity to discuss real implementations.

That’s why we need to not only prioritize it but also get better on it so we can extract its full value.

Getting better at it

There are two main things you can do to help your team with code reviews, two rules of engagement, the first rule is what you can do as an Author of a pull request and the second is what you can do as a reviewer.

As an author

If the content is king, context is God - Gary Vaynerchuck

This quote was about social media but it can also be applied to code reviews, if the content is your code then the context is the “Why” you are doing it.

Provide context is the main thing you should do as an author of a Pull Request, not only the context behind the “Why” you’re doing that change but also any technical context that might help other folks down the road, like the techniques you used, or maybe explaining a design pattern you choose. Let the reviewer know what you have learned in the process of doing this code, that’s a great opportunity to share knowledge.

As a Reviewer

The main rule for a good review process is

Ask, don’t tell.

Feedbacks on code reviews tend to be received much more negatively than when done face to face. So we need to overcome this if we want our feedback to be received in the right way.

One way of doing that is to offer compliments, I do recommend you to do it, make sure to point out the good stuff and the things you have learned from a PR.

The key to overcoming the negative tone a code review can have is to ask questions instead of making demands. Questions have the power to start healthy conversations.

Comments like “Extract this into its own function to avoid duplication” only provide two alternatives to the author, she either changes it or replies starting what looks like an argument. But instead, you could rephrase that for “What do you think about extracting this into its own function to avoid duplication?”, that opens up a bunch of possibilities to actually start a conversation, maybe she has thought about it already, maybe that is not possible, or maybe that is actually a good idea and she will embrace it and change the code.

“What do you think about” is a great conversation opener for code reviews.

Wrapping up

Code reviews can be a great tool that can empower you and your team, but as any tool, it needs to be used properly otherwise it can make people extremely uncomfortable to pitch in their ideas and to contribute more to the codebase.

Keep in mind that Code Review is so much more than finding bugs, it actually helps your company to spread knowledge and your developers to get better at their jobs.

Last but not least, this article was hugely inspired by a great talk of Rails Conf 2015 called Implementing a Strong Code-Review Culture so make yourself comfortable to check it as well and get more in-depth explanations and examples.

Unboxing Data Science, Chapter II

Sat, 02 Jun 2018 00:01:00 +0000

You might have read the first chapter of this series a few days ago, the idea was to see if that was something that people were willing to check before I invested too much time into writing this, but it seems people are eager to learn more about data science e see how that is applied to real businesses and real needs.

In this series, I’ll share how I tackled three different challenges and how I used data science to produce some great tools in the e-commerce project I was working back then, the challenges were:

How can I predict how much money we will make next month?
What customers clusters we have and what defines them?
Can I predict a customer quality as it puts its first order?

In this chapter, I’ll talk about the first point, how I was able to predict the e-commerce monthly revenue with a margin of error of 10%.

Tools

For this initial challenge I used numpy, pandas and sklearn, we will also use two different Machine Learning algorithms, the classic Linear Regression and one way more complex called Gradient Boosting Regressor, worry not, I’ll explain how each one works so bear with me.

I’ll start by importing everything I need:

# Importing necessary libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import GradientBoostingRegressor
%matplotlib inline
plt.style.use('fivethirtyeight')
plt.rcParams['font.serif'] = 'Ubuntu'
plt.rcParams['font.monospace'] = 'Ubuntu Mono'
plt.rcParams['axes.labelweight'] = 'bold'
plt.rcParams['xtick.labelsize'] = 15
plt.rcParams['ytick.labelsize'] = 15

Assesment

Starting from an empty Jupyter notebook I poked a bit around the orders data I had.

# =======================================================
# Read Orders data by day
# =======================================================
orders = pd.read_csv('orders.csv')
# Convert datetimes to pandas datetime type
orders["created_at"] = pd.to_datetime(orders.created_at)
orders["processed_at"] = pd.to_datetime(orders.processed_at)

The main goal of this part was to gain some domain knowledge, understand what were the main factors that drive the revenue for a specific month, those are the factors that we want to feed into the prediction mode.

The first thing that I noticed when plotting the orders by month was that there was some seasonality and that the month of the year has a huge influence on the revenue as you can see below:

# Getting paid Orders only.
# ============================================================
orders = orders[(orders.paid == True) & (orders.canceled == False)]
by_month_data = orders.month.value_counts().sort_index()
f, (ax1) = plt.subplots(1, 1, figsize=(25, 10))
plot = sns.barplot(by_month_data.index, by_month_data.values, ax=ax1)

So the month of the year is definitely a data that we need to feed into our prediction model.

I had only 3 years worth of data, but that was enough to check how much a month revenue correlates with all other months revenues:

from statsmodels.graphics.tsaplots import plot_acf
f, (ax1) = plt.subplots(1, 1, figsize=(25, 8))
plot_acf(monthly_data.revenue, ax=ax1)
pyplot.show()

Each point on this graph indicates how much one month correlates with the other months, axis X representing all months I have, 33 months, and axis Y representing the correlation index, here we can see that a specific month has a high correlation with the point 12 and 13, that indicates that a month’s revenue is highly correlated to the same month’s revenue one year before.

Basically what it means is that if you had a good revenue in January last year that might imply on a good January this year, the monthly revenue tend to follow a pattern. This means that the revenue from the same month but on the last year is another data that we need to feed into our Machine Learning model.

More Data

Now that I had some insight into the data I had it was time to put together other data that might have a say in predicting in predicting the revenue for a month.

After some poking around I found other data that was worth having: The Ads Investment ($) it was planning on doing that month.

Once I had all the data in hand was the time to apply to Machine Learning on it, the data I decided to use was:

Last year’s same month revenue
Last month’s revenue
$ spent on Ads for the current month
What month it is
What season it is

Trying Linear Regression

Known as one of the simplest ML algorithms, linear regression is pretty straightforward and rely on you having data that have a linear relationship.

I spare the last 4 months of my data so that I could test my prediction model and see if it would have guessed how much the revenue would be for those.

test_data = final_data[-4:]
final_data = final_data.drop(test_data.index)

X = final_data[feature_cols]
y = final_data.revenue
linreg = LinearRegression()
linreg.fit(X, y);

f, (ax1) = plt.subplots(1, 1, figsize=(20, 8))
X = test_data[feature_cols]
y = test_data.revenue
ndf = pd.DataFrame(y)
values = linreg.predict(X)
    
ndf['estimates'] = values
ndf['difference'] = ndf.estimates - ndf.revenue
ndf
plot = ndf.plot(kind='bar', ax=ax1);

The prediction was okay, but it wasn’t as effective as I wanted it to be, in some months, like November it missed by more than 30%, and that wasn’t as helpful as I needed it to be taking into account we wanted to use this intel to better prepare ourselves, to be more proactive than reactive in terms of the business in general.

So I decided I needed a better ML (Machine Learning) model that could better capture the relationships between the data that I had.

Trying Gradient Boosting Regressor

I decided I needed a model that could capture more complex relationships other than linear, and while looking for it I learned about the Gradient Boosting Regressor, it seemed awesome, but the issue is that it’s based on Decision Trees, and Decision Trees are good on predicting values that it already knows, but there is no way it could predict a value it has not yet seen, and that was exactly the case because we as an e-commerce startup were going through a huge growth so our numbers kept growing as well, and it would be impossible for the algorithm to predict a revenue higher than the ones it has already seen.

So that is when I learned about how to deal with trends and use a method called Random Walk. First, let’s check the trend over the past years of data:

P.S. the drop on the end of the chart is because it was missing the how of the last month worth of data.

This chart shows a trend in the number of orders, the same trend occurs on the revenue and the fact that we are always increasing the revenue is what makes it hard for us to use a Decision-Tree-based model, but what if instead of estimating the revenue we could estimate another value that is kept under a range? You got it, instead of estimating the revenue we can estimate the difference between the current monthly revenue and the same month’s revenue from the year before.

As you can see the difference is kept under a usual range, sometimes more and some less but within a range and that enables us to use a Decision-Tree-based model, so all we have to do is to estimate the difference and then sum it up with the revenue from the same month one year before.

Once I’ve trained the model this is the results I got for the same 4 last months of the year:

P.S. there was some tunning to figure out the best number of trees and the maximum depth of each tree.

f, (ax1) = plt.subplots(1, 1, figsize=(20, 8))
X = final_data[feature_cols]
y = final_data.revenue - final_data.last_year_same_month_revenue
X_test = test_data[feature_cols]
ndf = pd.DataFrame(test_data.revenue)
params = {'n_estimators': 1540, 'max_depth': 2}
est = GradientBoostingRegressor(**params).fit(X, y);
ndf['estimates'] = est.predict(X_test) + test_data.last_year_same_month_revenue
ndf['difference'] = ndf.estimates - ndf.revenue
plot = ndf.plot(kind='bar', ax=ax1)

Here is a comparative on how it predicts better when compared to a just simple linear regression, check how close the GradientBoostRegressor algorithm + Random Walk got when compared with the Linear regression.

As you can see the new algorithm performed way better and is still live in production producing results within a margin of error of 10%.

Once I had the prediction model it was only a matter of exporting it and making it an API so that I could integrate it with an dashboard.

This way the business can now predict the next month revenue and properly prepare for it.

Next Steps

I hope you have liked this article I tried to not make it too technical while also sharing some code so you can try out some cool stuff on your side as well, I’m not sure if I will be able to share all the code involved because of confidentiality concerns but I’ll make sure to share as much as I can.

If you liked this chapter make sure to let me know on Twitter, it makes a huge difference getting me pumped and willing to write more about it.

On the next chapter, I’ll talk about the second challenge: What customers clusters we have and what defines them?

Unboxing Data Science, Chapter I

Fri, 01 Jun 2018 00:01:00 +0000

Back when I was a teenager I learned a valuable lesson from my father that I kept with me through my career:

Never settle for untapped potential.

I can’t stress this enough, untapped potential freaks the shit out of me, I’m not only talking about personal potential but also businesses potential. With that as a guide I always tried to make sure that whatever project I’m working on, it should be enabling the people and business behind to use its full potential. And that is the kind of thought the made interested in Data Science.

Learning data science can be a pain for newcomers, and I believe the reason behind it is the lack of understanding the possibilities of using data science, for me that was a crucial turning point on passing the initial learning curve, understanding what value I could extract from it and how exactly I could use it.

My plan is for this article to be an initial kick off on a new series of articles I’m calling Unboxing Data Science, still deciding if I this is something I should invest the time into writing, so If you want to see it through let me know over twitter.

Getting started

I was working on an e-commerce and that was growing like crazy, revenue was going great but that was starting to raise some issues like, “How much money are we making next month” or “Should we prepare more inventory”, these are really important questions and put a business into reactive mode instead of a proactive mode.

That’s when I first decided to learn data science. Data science is all about prediction, you wanna try to build a model that will predict an outcome based on other data, so I decided that my first challenge would be to create a prediction model for the next month revenue.

The Stack

For this series, we will use Python, a Jupyter Notebook and a few libraries like panda, numpy and scikit-learn, we won’t use it all in this first article, but you will need to install Python 3 and Jupyter, an easy way to get all your environment ready is to download Anaconda, it will install all dependencies we need at once.

From Anaconda you can easily launch Jupyter and create your first notebook, I won’t dive into the details of that because it’s pretty straightforward and there are a ton of help online already.

The Data

Because of the stack we are going to use, the easiest way to consume data is in the form of a .csv file and that is exactly what I used, first I downloaded a dump and converted that into a .csv file.

As you learn data science you’ll realize that the key difference between building a great model is the data that you have and the new data you generate from it. That together with domain knowledge is key to build great models. In my case, I had all the data I need right in front of me, 3 years of orders histories right there in the e-commerce’s database, so I started poking it a bit first checking the number of orders by month.

That’s when we start to see some patterns, it’s clear that the sales build up on the lasts months of the year, so we can definitely see how the month itself impact the revenue therefore that is something we need to take into account on our model.

We can also see this pattern by season:

All that translates into data that we can use down the road when building our model. Another crucial component for me was to check if there is a trend for the revenue, that was key for me down the road as well:

Here we can see how as the time goes by the sales tend to grow, and that makes measuring future revenue even more crucial for the business itself.

Next Steps

Wanna know more about how I solve this and get deeper into learning Data Science and Machine Learning? If you do, let me know, I’m trying to decide if this series is something I should invest my time in producing and getting feedback is crucial for me to take this decision, you can do that through my twitter.

If that gets enough traction I’ll write the next article now getting technical and showing how I not only built this initial charts but also the next steps I took after that.

I'm Joining Toptal

Thu, 01 Mar 2018 00:01:00 +0000

Hey folks, I’m joining Toptal as Engineering Manager, and I’m super excited about it, also really excited to finally share this with you all!

For those that doesn’t know, Toptal is a network of freelancers that have the top software developers, designers, and finance experts on their pool. They are well known among developers and have a pretty challenging interview process.

I’m super thankful about the work I was able to do in the past years, had awesome opportunities, from being the CTO of a mid-sized Brazilian startup, to making the jump to the international market and working remotely for amazing startups both in NY and SF.

I had the opportunity to lead remote teams and also did a bunch of stuff I was really interested about, from learning and using Elixir as my main programming language, what I did for almost 3 years (currently writing a book about it), to doing an specialization on Data Science and using it in the real world.

While doing all that I had the opportunity to meet a ton of awesome people and talk on great events.

Through it all I always counted on my habilities to help and interact with people, that made a key difference on my carrier in general and prepared me to tackle different challenges.

A few months ago I decided I was familiar enough with the processes of working remotely so that I could go back to a more managerial role, where I can add more value and help other people to tackle the challenges they are facing, while they are shaping their careers.

That was my main motivation on joining Toptal, knowing that the have really talented people and a ton of awesome challenges to tackle.

Building a Mobile App with Ember + Cordova + Rails

Fri, 02 Feb 2018 00:01:00 +0000

You wanna build that great side project you have been thinking about, but you have never built a mobile app before, fear not my friend, bear with me through this article and you will be amazed about how fast you can take your idea of the ground.

I did write an article in the past about building mobile apps with ember and Cordova, maybe you even read it already, that was a gentle quick off on how to quickly build an app, so I decided to write a new and better version of it.

The Stack

For this article we will be using Ember, Cordova and Rails, you might be wondering why, the reason in simple, because that is the fastest stack to build an mobile app I’ve tried in the past years.

I’ve done native, react, ionic, I have tried it all, but this combo still is the fastest one to develop in. Keep in mind that I’m a believer of the “move fast and break things” mantra, and that the main goal here is to spin off an idea out of the ground, not necessarily achieve the best performance. This stack will put you in that sweet spot of building something fast, using tech that you might already know, and avoid boilerplate code.

Installing

Start by installing the tools we will use, I won’t spent much time explaining this, there is a bunch of great guides around.

Postgres
Ember.js - npm install -g ember-cli
Cordova - npm install -g cordova
Corber - npm install -g corber
Rails (Assuming you have Ruby installed) - gem install rails

Creating the Application

In this article we will create a product listing app, that will be fed by an Rails API, with a simple CRUD admin.

Let’s start but creating a new ember application and initiating corber

ember new my_freaking_awesome_app
# ...
cd my_freaking_awesome_app
corber init

Corber will guide you through its setup process, letting you choose the platforms you want (iOS, Android, or both).

Now we have a brand new ember app to star to work with, if you want to fully understand ember you can check its guides, but for now we will use two of the ember “features”, routes and a components.

Let’s start by making sure our app shows its awesome name on its first view, go to the file app/templates/application.hbs, remove the {{welcome-page}} and add the proper title before the {{outlet}}:

<h1>Freaking Awesome App</h1>
{{outlet}}

Creating a Route

Now let’s create an route for our app, for the sake of this article lets say our awesome app is a shop list, with a list of products we need to buy on our next run to the groceries store, so lets start by creating an route to show the products we need.

ember generate route products

Open the new generated template at app/templates/products.hbs and add a proper title to it:

<h2>Products</h2>
{{outlet}}

Let’s make this our root route, so update the router.js to define that:

import EmberRouter from '@ember/routing/router';
import config from './config/environment';

const Router = EmberRouter.extend({
  location: config.locationType,
  rootURL: config.rootURL
});

Router.map(function() {
  this.route('products', { path: '/' }); // <- This is it!
});

export default Router;

Running our App

Let’s first run our app on the browser so we can check how it’s looking so far before compiling it to our phones. To do it just run:

ember serve

Now you should be able to access http://localhost:4200 on your favorite browser and see how it’s looking so far.

Create a Component

Now let’s create our first component, that will hold our product list:

ember generate component product-list

We can open our component template at app/templates/components/product-list.hbs Let’s add a loop to show all of our products. Replace the content of the template with:

<ul>
  {{#each products as |product|}}
    <li>
      {{product.name}}
    </li>
  {{/each}}
</ul>

Now lets go back to our template for the products route at app/templates/products.hbs and load our new product list component in there replacing the {{outlet}} with:

{{product-list products=model}}

Now we need to define the model, on the route itself, and feed that with our Rails API.

Create Product Model

Let’s use the ember generator to create the Product model on ember’s side.

ember generate model product name:string

With our model created, now we can set it to find all products on the product router we created before, just create a new model function at app/routes/products.js like bellow:

import Route from '@ember/routing/route';

export default Route.extend({
  // Declare the function bellow, its simply getting all products (that will come from rails)
  model() {
    return this.store.findAll('product');
  }
});

Integrating Ember with Rails

Now we need to set an adapter that will properly tell our ember app to call the rails app whenever looking for products.

Let’s start by creating an Adapter, that will inherit one of the existing ember adapters.

Create a new folder adapters under app and a file application.js inside of it with the following content

import DS from 'ember-data';

export default DS.RESTAdapter.extend({
  host: 'https://ember-app-article.herokuapp.com/', // Address of our rails app, link to the rails app
  buildURL: function(record, suffix) { // hack to mount the url correctly to our case
    var s = this._super(record, suffix);
    return s + ".json";
  }
});

Create Rails App

Disclaimer: In case you want to skip this, you can use an application I already created for it online at https://ember-app-article.herokuapp.com/

This is meant to be pretty straightforward, you can learn more about rails on its website and on other great tutorials, I’ll assume you have installed all you need, so lets just open a new terminal and create a new application:

rails new product_list_admin -d postgresql
# ...
cd product_list_admin

Now let’s create a scaffold for our products by using rails generators and running migrations:

rails g scaffold product name:string
rake db:create
rake db:migrate

Now you can start your rails application by running:

rails server

And you will be able to see our new products management interface at http://localhost:3000/products. Through this interface you are able to add new products to your list.

Now we need to make sure the Rails app is ready to accept calls from the ember app, so lets add rack-cors to our project and enable it to accept calls from our ember app.

Enabling Cors

Disclaimer: This would allow your app to receive requests from any other address and isn’t the safest option, recommend you to understand CORS better before using this in production.

Add rack-cors to your Gemfile:

gem 'rack-cors'

Update your config/application.rb to include the following:

 config.middleware.insert_before 0, Rack::Cors do
   allow do
     origins '*'
     resource '*', :headers => :any, :methods => [:get, :post, :options]
   end
 end

Updating Products Controller

Because ember expects us to send the products in a specific format we will need to do a quick update on our ProductsController to make sure we return a root node on the JSON response.

Just update your index function on the ProductsController to the following:

def index
  @products = Product.all
  respond_to do |format|
    format.html { render :index }
    format.json { render json: {products: @products} }
  end
end

This just makes sure whenever returning the JSON version of the index route it will have a root node product.

Deploy Rails App

I won’t spend much time explaining the ins and outs of deploying a rails app, but I do recommend you to deploy it in heroku for now, just for the sake of simplicity. Also make sure to update the ember adapter we declared before with the url for your deployed rails app.

Compile our app

If you run your ember app and rails app you will now notice your products will appear on the ember app, and you can add and edit them over on your rails app.

Let’s compile and test our app on a simulator. Depending on the platform you are building for (android or iOS) you will need to install different stuff, I won’t go over it all, but basically you will need Android Studio or Xcode (only Mac), I’ll assume you have installed it already, so go into your ember project folder you can run.

corber build --environment=production --platform=ios

Once that is compiled you can then open the project on Andoird Studio or Xcode by running:

corber open

From there you can build that on a simulator or on your on phone, this is how our awesome app should look like:

Wrapping up

This was supposed to be more of a hands on approach to quickly building mobile apps using overall web knowledge, I invite you to understand how Ember and Rails works better, so you can start to take more advantage of these framework as you build your apps.

If you liked this article please let me know over twitter, my handle is joaomdmoura, and I’m always eager o engage in nice conversations.

State Machine in Elixir with Machinery

Tue, 02 Jan 2018 00:01:00 +0000

State machines are a well defined concept that developers all around have been using, some without even knowing it.

Finite State Machines

Finite state machines are basically a control flow, a sequence of states and a define set of rules about the transitions between those states.

One of the best examples of that are shopping carts, it has pretty well defined states, and a set o triggers to transition it from one state to another.

Let’s check an example bellow:

As detailed on the imagine above we can see a set of defined states:

Empty
Filled
Payed
Abandoned

Each transition is triggered by a specific function, but there is even more into it, some transitions can trigger other callbacks, or even have guard conditions, let’s see how that looks once we put it all together:

Okay, that escalated quickly. As you can see, once you include the callbacks (green) and the guard conditions (blue) the logic starts to get more complex and the code can start to get messy.

After an initial implementation, as the state machines gets more complex, some logic will probably start to be split and fall all over the place, there will be callbacks on controllers, some as private functions some not, maybe some guard conditions will end up as validations on models, well you can get the whole picture, it’ll soon became unsustainable to keep extending it. Some other developers might go another routes and just put it all on the model, what some might consider an even worse bad smell.

So the question remains: how to implement useful state machines on Elixir without a bunch o boilerplate and/or messy code?

Machinery

Machinery is an open source library I’ve been working on the last months, it’s a thin State Machine library that integrates with Phoenix out of the box, but you can use it in any Elixir application, even not a Phoenix one.

It’s just a small layer that provides a DSL for declaring states, having guard clauses and callbacks for structs in general. It also has (when implemented with Phoenix) an optional build-in GUI that will represent each resource’s state.

Machinery is built to work with any structs and also offer you the ability to have different states machines.

How to use it

The package can be installed by adding machinery to your list of dependencies in mix.ex

def deps do
  [
    {:machinery, "~> 0.12.1"}
  ]
end

Be sure to also add a new field to the struct that will have the state, you want to make sure that is a string. If you have a Phoenix app it will involve create a new migration and changing the model accordingly.

Declaring States & Transitions

After that, you can create a new module to hold your state machine logic, in there you will declare your states, valid transitions, callbacks and guard conditions.

You will later down the road pass this module as an argument to be used when changing states.

defmodule YourProject.UserStateMachine do
  use Machinery,
    # The first state declared will be considered
    # the initial state
    states: ["created", "partial", "complete"],
    transitions: %{
      "created" =>  ["partial", "complete"],
      "partial" => "completed"
    }
end

Transitioning to new state

After having that in place you can transition state from anywhere on your application using Machinery.transition_to/3, it expects three arguments:

The struct that holds the state
The state machine module we just declared
The state we want to transit it to

Machinery.transition_to(your_struct, UserStateMachine, "next_state")
# {:ok, updated_struct}

That is just the initial implementation, let’s not forget that you can still declare the callbacks and guard functions, let’s check how to implement those using machinery.

Guard Functions

Guard functions or guard conditions, are functions that you can declare inside your state machine module, it’s expected to return a boolean value, if true it will let the transition occur and if false it’ll block it.

As you might notice, you should use the second argument to pattern match what state you want to guard.

# Guard the transition to the "complete" state.
def guard_transition(struct, "complete") do
  Map.get(struct, :missing_fields) == false
end

Before and after callbacks

Before and after callbacks are pretty straightforward, you should declare it inside the state machine module as well, it’ll execute any code inside this function before or after the transitions occurs, it’ll receive the struct as argument and it’s expected to return the struct back.

Here as well, you should use the second argument to pattern match into what state transition you need to trigger the callbacks.

# callbacks should always return the struct.
def before_transition(struct, "complete"), do: struct
def after_transition(struct, "complete"), do: struct

Persisting state

Keep in mind that Machinery won’t update your database automatically, but it can if you tell it how (because that may depend on what you’re using to store your data). The way you are expected to do it it through a function called persist/2:

persist/2 should always return the updated struct.

defmodule YourProject.UserStateMachine do
  alias YourProject.Accounts

  use Machinery,
    states: ["created", "complete"],
    transitions: %{"created" => "complete"}

  def persist(struct, state_you_are_transiting_to) do
    # Updating a user on the database with the new state.
    {:ok, user} = Accounts.update_user(struct, %{state: state_you_are_transiting_to})
    # Returning the User struct as it should.
    user
  end
end

The shopping cart flow

Let’s check a real case scenario and implement the initial cart behavior described on the flow charts I shared earlier and see how that would look like:

defmodule FakeProject.ShoppingCartMachine do
  # We start by declaring all states and 
  # each permitted transition.
  use Machinery,
    states: ["empty", "filled", "payed", "abandoned"],
    transitions: %{
      "empty" =>  "filled",
      "filled" =>  ["payed", "abandoned"]
    }

  def guard_function(cart, "filled") do
    # Check if there is enough of this item in stock
    # it returns a boolean, if true it will move on with
    # the transition, if false, it'll block it and keep
    # the previous state
    Item.has_stock?(cart.item)
  end

  def guard_function(cart, "payed") do
    # Check if payment is received and return boolean
    # if it returns true the transition will be allowed
    # to happen
    Payment.status(cart) == :confirmed
  end

  def before_transition(cart, "filled") do
    # A transition callback that will perform an action,
    # in this case right before the transition occurs,
    # locking the items on a cart to prevent another
    # customer from adding it if there is not enough stock.
    Item.lock_form_cart(cart)
    cart
  end

  def after_transition(cart, "abadonned") do
    # An after transition callback, it's used to perform
    # an action after a transition.
    # In this case Unlocking the items on the cart just 
    # moved to the abandoned state.
    Item.unlock_form_cart(cart)
    cart
  end
end

And there it is, we have a simple and straightforward implementation of a state machine, using callbacks and guard functions with a nice and readable DSL.

Now we can easily create transition it from one state to another making sure all guard and callbacks functions are properly handled.

BONUS: Dashboard for your State Machine

If you’re using Phoenix and want a visual dashboard representing your state machine (its states and each resource), you can easily have it. It will also enable you to change states by dragging it from one state to another. (pretty much like Trello)

This is how it looks like:

To enable the Machinery Dashboard all you need is to:

Add the plug Machine.Plug to you Endpoint module. ```elixir defmodule YourApp.Endpoint do # …

# It accepts the path you want to mount the dashboard at as an argument, # it will mount it under /machinery as default. plug Machinery.Plug # plug Machinery.Plug, ‘/my-custom-route’

# … end

2. Add the proper config to `config.mix`
- **interface:** a flag to enable the dashbord.
- **repo:** your app's repo module.
- **model:** the model that will hold the state.
- **module:** the machinery module where you have the declared states.
- *(Optional)* **dashboard_states:** A list of the states you want on the dashboard.

```elixir
config :machinery,
  interface: true,
  repo: YourApp.Repo,
  model: YourApp.User,
  # Optinal: dashboard_states: ["created", "partial"],
  module: YourApp.UserStateMachine

That’s it, now you can start you Phoenix app and navigates to http://localhost:4000/machinery, or whatever custom routes you have mounted the dashboard at.

Wrapping up

I hope you have liked the article and have fully understood what is a state machine, how it works, and how you can easily build one into your elixir project using machinery, even if it isn’t a Phoenix app.

I’d love to hear a feedback from you, so please let me know on the comments bellow and over twitter.

You can also check and help the Machinery project by checking it on Github.

Learn Elixir with a Rubyist, episode VII

Mon, 09 Jan 2017 00:01:00 +0000

Processes Communication

Hey folks, I wonder how many of you have read all episodes so far, if you are one of them please shout out to me on twitter, I’d love to have some feedback :)

This is a series of short bar-like conversations around Elixir and its features, it aims to help you to wrap your head around it by using meaningful examples and putting it in the context of real world problems.

If you haven’t yet, you probably should check the last episodes, I’ve been writing this in a logical sequence that makes it way easier to understand by gracefully increasing the complexity of the topics and examples:

Processes Communication

We have talked a lot about processes on other episodes, not only how they work but also how they are a key component of Elixir, something it inherited from Erlang, and a core piece to extract the most out of the language capabilities.

On Episode II - Actor Model, Modules and functions we talked about Actor Model, a known architecture that was used on the implementation of Erlang VM, it enables processes to be totally isolated from each other, not sharing any context, and by doing so, it actually makes easier to share specific information between processes, but in a organized way, by exchanging messages.

Every process in Erlang (and therefore in Elixir as well) is considered an Actor, completely isolated from its pears, a single processes might be related to other processes, like Supervisors, or linked processes (we’ll talk about that in a near future), but it does not share the same context and variables with other processes.  

Every process also has what you could imagine as an empty mailbox, just waiting for messages, and you can program your processes to respond to those messages depending on their content. You can also make your process send messages to others by using an identifier, a process ID, what we call PID (Process Identifier).

This alone can sound silly, but having this ability is a huge deal and enables you to do a lot of fun stuff you wouldn’t do in other non-functional languages like Ruby.

Let’s start by checking on how to send, receive and respond to messages on processes, doing this will involve some previous knowledge from past articles mostly episodes I, III and V.

# Let's start by figuring out our console Process ID (PID)
# by now you already now that all you need to strat the 
# interactive console is to type `iex` on your terminal.


# The `self/0` function return the current Process PID.
current_proc = self()
# #PID<0.56.0>


# Now we can use the `process_info/2` function 
# from Erlang to get all messages our current 
# process (the interactive console) have
:erlang.process_info(self(), :messages)
# {:messages, []}
# As you can see, there is no messages waiting.


# We can send a message from this process to itself by
# using the `send/2` function, it has two arguments, the 
# PID and the message.
send(current_proc, "new message")


# Now, if we check our current process messages we will
# find one message waiting to be dealt with.
:erlang.process_info(self(), :messages)
# {:messages, ["new message"]}


# To handle messages we need to use a receive block
# it's really straightforward and uses pattern matching
# agains the next message on the list.
receive do
  message -> IO.inspect("Received #{message}")
end
# "Received new message"
#
# The message is displayed and now if we check the list
# of messages, it should be empty again

:erlang.process_info(self(), :messages)
# {:messages, []}

As you can tell, sending and receiving messages is something really simple in Elixir, and that’s one of its major benefits, it makes super easy for you to take advantage of its main capabilities.

In this first example we send only a string, but keep in mind you can send any type of data, the most commonly used are tuples (you can check more about tuples on Episode IV - Elixir Types, Data Structures and Underscore).

Another thing that you might have noticed is that the receive block only handles one message, the next one on the list of messages. If you have multiple messages waiting, the receive block will only handle the next one, the one on the bottom of the list:

current_proc = self()
# #PID<0.56.0>


:erlang.process_info(self(), :messages)
# {:messages, []}
# No messages waiting.


# This time we sen multiple messages to this same process
# we will send tuples, with a `:new_message` atom and
# an integer
send(current_proc, {:new_message, 1})
send(current_proc, {:new_message, 2})
send(current_proc, {:new_message, 3})


# On our messages list we'll find all three messages
:erlang.process_info(self(), :messages)
# {:messages, [new_message: 1, new_message: 2, new_message: 3]}


# Now we use the `receive` block to pattern match the
# tuple with `{:new_message, n}`, and then display a message
# indicating we got the message
receive do
  {:new_message, n} -> IO.inspect("Received new message: #{n}")
end
# "Received new message: 1"
# 
# Only the first message was handled, the other two
# are still in the list waiting for another 
# receive block

:erlang.process_info(self(), :messages)
# {:messages, [new_message: 2, new_message: 3]}

Okay, but how can we have a process constantly waiting for messages and handling it as it arrives? That’s where recursion comes into play, we talked about it on Episode V - Concurrency, Processes and Recursion, and this is an awesome use case for it.

# Let's create a `Sum` module that will have a `sum_number/1`
# funciton.
defmodule Sum do

  # The `sum_number/1` expects one argument, an integer 
  # that will be used to perform a sum operation  
  def sum_number(x) do

    # Here we use `receive` to match messages into the
    # tuple `{:sum, n}` where `n` is an integer sent
    # within the message.
    # At this point the process will stop and wait for a 
    # message (if there is none) before trying to match it
    # and move on.

    receive do
      {:sum, n} -> 

        # We display the result of the sum between the number 
        # passed as argument to `sum_number/1` and the number
        # on the received message
        IO.inspect(x + n)

        # Then call this function again passing the same argument.
        # By using recursion (calling this function again)
        # we setup the receive block again, waiting for the 
        # next message, so every time a message is received
        # this function will handle it and setup a new receive 
        # block to handle the next one.
        sum_number(x)
    end
  end
end

# Now we spawn a new process to execute the `sum_number/1`
# function from the `Sum` module we defined above, passing `2`
# as the argument, and this will return the PID for this new process
sum_proc = spawn(Sum, :sum_number, [2])
# #PID<0.121.0>


# Then we can send a message for the process we just spawn above
# passing the tuple `{:sum, 3}` to check what will be displayed
send(sum_proc, {:sum, 3})
# 5
#
# The number `5` is displayed as expected result of `2 + 3`.
# Because the process uses recursion we can still send messages
# to it and it stills alive, waiting for another message.

send(sum_proc, {:sum, 10})
# 12

send(sum_proc, {:sum, 6})
# 8

send(sum_proc, {:sum, 2})
# 4

# Here we can check the process is still alive and 

# therefore waiting for messages
Process.alive?(sum_proc)
# true

Now we already know how to use recursion for creating a process always ready to receive messages, but what about multiple processes communicating with each other? Let’s check bellow.

Practical Example

For this example we’ll build a in-memory storage for gravatar images and its paths. Before diving into the code it’s important that you understand that functional languages do not have the usual structures for holding state, keep in mind there is no objects nor instances. Recursion is one of the ways you can support state, and that’s exactly what we need for this example.

We want to have an module with a function that returns the local path for a gravatar image for a given email, in order to achieve that we will need to perform the following actions:

Check if we already have this image locally
Check if the image on Gravatar
Download the image and save it locally
Return the local image path
Append the new image path to a in-memory storage

# We start creating our Gravatar module
defmodule Gravatar do

  # We'll get into `ensure_all_started/1` on future episodes
  # but here we have to start `:inets` app because we'll
  # need one of its modules down the road when downloading
  # the gravatar image, `:inets` is part of erlang 
  # standard library
  Application.ensure_all_started(:inets)

  # This is our main function `gravatar_images/1`, it expects
  # a `Map` as argument and this `Map` will store the paths
  # for all images as we find and download them.
  # As you can tell we are using pattern matching to ensure
  # the `images_path` argument will be `Map`, this doesn't
  # mean it needs to be empty, we could start with pre-existing
  # images already.
  def gravatar_images(images_path = %{}) do

    # Here is our receive block, it tries to match messages
    # into three different patterns (all expect to receive
    # also the PID from the process that sent the message): 
    #
    # - {:all_stored_images, pid}
    # - {:get_image, email, pid}
    # - {:store_image, email, path}
    receive do

      # `:all_stored_images` will basically send a message
      # back for the process that sent this, with all the
      # images, by returning the `images_path` map. 
      # Then it will call `gravatar_images/1` again, passing
      # the same map as argument once again, so we sill have 
      # the receive block ready for the next message.
      {:all_stored_images, pid} ->  
        send(pid, images_path)
        gravatar_images(images_path)

      # `:get_image` will send a message back to the PID that
      # send this message with the local path for that image
      {:get_image, email, pid} ->  

        # `get_image/2` is a private function defined down bellow,
        # it tries to find the image into the Map we are using
        # as storage, if it doesn't finds it, it'll download
        # the image and return the path
        new_path = get_image(images_path, email)

        # Here we send the message back with the `new_path`
        # of this image to the process that sent the 
        # initial message
        send(pid, new_path)

        # Then we send a message to this own process but
        # with the tuple `{:store_image, email, new_path}`
        send(self(), {:store_image, email, new_path})

        # We call `gravatar_images/1` once again because we
        # need to setup a new `receive` block to handle the
        # message we just sent above with the `:store_image` tuple.
        gravatar_images(images_path)

      # `:store_image` will update the email key on the map
      # we are using as storage to store the local path for
      # the image.
      # Using the pipe operator we call `gravatar_images/1`
      # function again, but now passing the new Map as the
      # argument, and that's what enable us to hold state
      # and having a in-memory storage.
      {:store_image, email, path} ->  
        Map.put(images_path, email, path)
        |> gravatar_images
    end
  end

  # The functions bellow are related to the gravatar 
  # integration, I'll avoid getting deep into explaning 
  # what is going on because it's not the main subject for
  # this example
  defp get_image(images_path, email) do
    # Tries to get the `email` key from the map, if there
    # is no key with that value, then call `download_image`.
    # Here we are using case with pattern mathing, we talked
    # about it in later episodes already.
    case Map.get(images_path, email, :not_found) do
      :not_found -> download_image(email)
      path -> path
    end
  end

  # Request, download and write the file to our local env
  # return the path as the function response.
  defp download_image(email) do
    path = "gravatar/#{email}.jpg"
    file = gravatar_file(email)
    File.write!(path, file)
    path
  end

  # Returns the binary of the image from gravatar API.
  defp gravatar_file(email) do
    http_opts = [body_format: :binary]
    url_data = {gravatar_url(email), []}
    
    {:ok, resp} = :httpc.request(:get, url_data, [], http_opts)
    { {_, 200, 'OK'}, _headers, body} = resp
    body
  end

  # Mounts the gravatar API url accordinly to 
  # their docs.
  defp gravatar_url(email) do
    email_hash = :crypto.hash(:md5, email) 
    |> Base.encode16(case: :lower)
    'http://www.gravatar.com/avatar/#{email_hash}.jpg'
  end
end


# First we spaw a new process, that will execute
# the `gravatar_images/1` method from the `Gravatar`
# module we just created, seding and empty map (`%{}`)
# as argument (so no image yet).
pid = spawn(Gravatar, :gravatar_images, [%{}])

# Now we can send this process a message to get the 
# avatar picture for a email
send(pid, {:get_image, "me@learnelixir.com", self()})

# If we check the messages our current interactive
# console has, we are supposed to have a new one with
# the local path for the image we just requested
:erlang.process_info(self(), :messages)
# {:messages, ["gravatar/me@learnelixir.com.jpg"]}
#
# As you can see, we do have a new message with the 
# local path for that image

# If we ask for all images stored so far it will
# shoot a new message back, now with the whole map
send(pid, {:all_stored_images, self()})

:erlang.process_info(self(), :messages)
# {:messages,
#  ["gravatar/me@joaomdmoura.com.jpg",
#   %{"me@joaomdmoura.com" => "gravatar/me@joaomdmoura.com.jpg"}]}
#
# Now we have two messages, the last one and a new 
# one with a map that has an email as a key and the
# path as value.

P.S.1 - If you already know Elixir and Erlang you probably noticed there is some room for improvement, I could have used the Task module or even ETS + GenServer to keep state. But for the sake of the simplicity of this examples it’s more than enough, we will talk about the Task module next, and about ETS a GenServers in the future.

What’s Next?

Finally on Episode VII of Elixir with a Rubyist! Again I failed to get into the Task module, but I do now we have gone through all the knowledge necessary to understand and use it, so the next episode will be dedicated to it!

I’m really glad from all the feedback I got so far, but I’d love to hear it from you, so please let me know on the comments bellow and over twitter.

If you don’t want to miss the next episodes and cool information around talks and even access to the code we use on the episodes you should subscribe to our mailing list by putting your email bellow.

On the next episode we will refactor our practical example using the Task module and understand how to use it.

Learn Elixir with a Rubyist, episode VI

Sun, 08 Jan 2017 00:01:00 +0000

[Head | Tail] & List Comprehension

Another week another article, glad to be back to publish one article a week, this is the sixth episode and as promised we will talk about one of the most used features of Erlang, the [head | tail] implementation and list comprehensions. Now is also a great opportunity to let me know what you’re thinking about the series on twitter and check the latest episodes:

Lists & [Head | Tail]

We have already talked about lists in the past articles, mainly on the - Episode IV - Elixir Types, Data Structures and Underscore, by now you may have realized that list are one of the main types in Elixir (and also in Erlang), because of that, there is a bunch o optimizations and tools that were created to support it.

Having a good support for lists is a common trait of functional languages because its tools enable you to achieve recursion easier, even without specific loop functions, as we already discussed on the last Episode V (Concurrency, Processes and Recursion).

[head | tail] was already present on Erlang, it enables you to split a list by taking away its first element and putting the rest of the list into a new list. Let’s start by checking some examples of how [head | tail] works:

# Declaring a list is really simple and straightforward,
# it can contain anything, from integers to other lists.
list = [1, 2, 3]

# Let's start by simply using pattern matching with the
# the [head | tail] syntax.
[h | t] = list


# You can see the value of `h` is now the first element
# of the list we had
h
# 1

# And `t` is the other elements from the same list but now
# into a new one.
t
# [2, 3]

# You can also use the Erlang `hd/1` and `tl/1` functions
# both take a list as argument and return its head or tail

hd(list)
# 1

tl(list)
# [2, 3]

# You can also use the [head | tail] syntax to push elements
# into a list, here `:atom` is being assigned as the head of 
# a list, and our old list as the tail, therefore it will 
# result in a new list with `:atom` as its first element

new_list = [:atom | list]
# [:atom, 1, 2, 3]

It may no seem but the head and tail implementation enables you to do awesome stuff, we checked some examples on the last episode, and will have event better ones on the next, but before jumping into practical examples let’s check more about list and its capabilities.

Lists Comprehensions

List comprehension is an old concept that is also present on Erlang, it can be used to construct lists (normally based on other lists) in a very natural, easy way, like a mathematician is used to do. It’s another common trait of functional languages because it enables you to loop through lists, filtering and mapping its elements.

Elixir brings a syntactic sugar for comprehensions, the for syntax , you can do most of common tasks using it, let’s check some examples of how it works:

 # The `for` syntax is pretty simple, it can be divided 
# into three different parts: an Enumerable, in this case
# a list `[1, 2, 3, 4]`, the variable to which each 
# elements is being assigned to (`i`), and the operation that
# will be performed using each element (`2 * i`).

for i <- [1, 2, 3, 4], do: 2 * i
# [2, 4, 6, 8]
# 
# As you can see each element of the list is being assigned
# to `i` and then multipled by `2`, resulting into a new list
# `for` also enable you to have filters applied to
# your comprehensions, let's check bellow:

for i <- [1, 2, 3, 4, 5], i > 2, do: 2 * i
# [6, 8, 10]

# As you can see, becasue we are now passing another argument,
# a filter (`i > 2`), elements that do now pass throught it
# won't be processes and therefore won't be on the new list.
# `1` and `2` were ignored and the numbers bigger than `2`
# were doubled as expected.

Lists Comprehensions + Pattern Matching

You can also put for together with pattern matching, by now you probably realized you can use pattern matching with almost anything on Elixir, by doing it we’ll be able you to filter the elements from the list (or other Enumerable).

This can be really useful when working with tuples (we talked about it on - Episode IV - Elixir Types, Data Structures and Underscore and multiple comprehensions. Let’s check how it works when using a list of tuples and pattern matching:

# We just create a list of tuples. In this case it 
# represents cities and its countries.

cities = [
  {"us", "new york"}, 
  {"us", "san francisco"}, 
  {"cn", "beijing"},
  {"fr", "paris"}
]

# Here we are using pattern matching by trying to match
# each tuple into the following pattern `{:us, city}`.
#
# The elements that doesn't match it will be ignored
# and the ones that do match will have its city name
# assigned to the `city` variable.

for {"us", city} <- cities, do: String.capitalize(city)
# ["New york", "San francisco"]

# As you can see only the first and second elements from
# our list were used and capitalized, because it was 
# the only ones that matched our pattern.

# Because we are using pattern matching we can do it 
# both ways, by also trying to match the city's name

for {country, "paris"} <- cities, do: (country)
# ["fr"]

Lists Comprehensions + into option

Another great and really useful feature that comes with the for implementation is the into option, it enables you to have the end result from the comprehension into another type that is not a list, like a map (we talked about maps on - Episode III - Maps, Functions + Pattern Matching = ❤️), here is a quick examples of how it works:

# Here we have a list with valid and invalids 
# responses, we want to get the error on that
# list and put it into a map, using the `error`
# key.
list = [
  {:ok, "valid"}, 
  {:ok, "valid"}, 
  {:error, "invalid argument"}
]


for {:error, error} <- list, into: %{}, do: {:error, error}
# %{error: ["invalid argument"]}
#
# by returning a tuple the `for` will understand 
# the the first element represents the key and the
# second one the value that should be on the map.
# And there is our Map, exactly how we wanted it.

I know it looks good enough already but gets even better, for also enables you to pass as many generators and filters as you need, check it out:

# This time we have 2 lists, one with integer 
# and another with atoms, lets use `for`
# on both at the same time
list_1 = [1, 2, 3]
list_2 = [:a, :b, :c]

for x <- list_1,
    y <- list_2 do 
      {x, y}
    end
# [
#  {1, :a}, {1, :b}, {1, :c}, 
#  {2, :a}, {2, :b}, {2, :c}, 
#  {3, :a}, {3, :b},{3, :c}
# ]
#
# The result will be the based on the comprehension 
# of both lists, getting a element from the first
# and going through all elements of the second one.

# You can have as many list you need, this concept
# also enable you to use as many filters as you want
# and even assign variables.

# In this example we will have two list, and we want to
# mutiply its elements by each other and puts the 
# result into a new list, but only if the result of the
# operation is bigger than `4`

list_1 = [1, 2]
list_2 = [3, 4]

for x <- list_1,
    y <- list_2,
    z = x + y,
    z > 4 do 
      z
    end
# [5, 5, 6]
#
# This is it! This example performed the following
# operations:
#
# 1 + 3 = 4
# 1 + 4 = 5
# 2 + 3 = 5
# 2 + 4 = 6
#
# But the result from the first calulation got excluded
# from our final result because it didn't pass 
# the `> 4` filter we had.

What’s Next?

Yay! Episode VI of Elixir with a Rubyist! I was hoping to get into the Task module into this episode but I thought it would be better to wait for the next one taking into account we had so much to check on lists.

This is a series of short bar-like conversations around Elixir, it’s aimed to help developers that are trying to understand and wrap their heads around it.

I hope you have liked it so far, if you do, please let me know on the comments bellow and over twitter.

On the next episode we will finally get into the Task module and understand how exchange information between processes.