Over the last few years, we've deployed a lot of environments into the AWS cloud. Provisioning a brand new IT environment requires a well planned out design, and each design is going to be different based on your business needs. But one aspect that has to always be present, from the original concept all the way to the execution, is security. These days security cannot be an afterthought. There's too much data, money and power riding on our modern IT systems, where getting a virus or having a hacker breach those systems is no longer an incoveniance, it can be a career ending event. This is why we always make sure to follow industry best practices when it comes to security. A lot of people believe security is a digital, black and white concept. Either you're secure or you're not. But in reality, you can never be truly fool proof. Security comes in layers, and the more layers you put in place, the more secure your environment is. In this post, I'll describe some of the
The Dendory Capital Datalab creates solutions, workflows, tools and pipelines for our clients' Big Data needs. As part of this process, we need to make sure the solutions we provide our clients work properly. Today, we're going to go over a simple use case of analyzing a dataset to gain useful insights about a particular problem. We're going to load a CSV file containing data from the NASA Near Earth Objects project, and try to find out whether or not any large object is going to come close to the planet in the next week. Singing up for Databricks Databricks is the commercial version of Apache Spark, and provides a handy web-based interface to create and manage clusters, start a notebook, and use Python code without any administration overhead. Better yet, they have a community edition we're going to be able to use for free. So the first thing to do is go to databricks.com and signing up for a community account. Once you confirm your email address, you can log into