Dendory Capital - Blog

Over the last few years, we've deployed a lot of environments into the AWS cloud. Provisioning a brand new IT environment requires a well planned out design, and each design is going to be different based on your business needs. But one aspect that has to always be present, from the original concept all the way to the execution, is security. These days security cannot be an afterthought. There's too much data, money and power riding on our modern IT systems, where getting a virus or having a hacker breach those systems is no longer an incoveniance, it can be a career ending event. This is why we always make sure to follow industry best practices when it comes to security. A lot of people believe security is a digital, black and white concept. Either you're secure or you're not. But in reality, you can never be truly fool proof. Security comes in layers, and the more layers you put in place, the more secure your environment is. In this post, I'll describe some of the

 The Dendory Capital Datalab creates solutions, workflows, tools and pipelines for our clients' Big Data needs. As part of this process, we need to make sure the solutions we provide our clients work properly. Today, we're going to go over a simple use case of analyzing a dataset to gain useful insights about a particular problem. We're going to load a CSV file containing data from the NASA Near Earth Objects project, and try to find out whether or not any large object is going to come close to the planet in the next week. Singing up for Databricks Databricks is the commercial version of Apache Spark, and provides a handy web-based interface to create and manage clusters, start a notebook, and use Python code without any administration overhead. Better yet, they have a community edition we're going to be able to use for free. So the first thing to do is go to databricks.com and signing up for a community account. Once you confirm your email address, you can log into

Taking notes is something that anyone who works with intellectual matters has to do. You typically start in high school, when the things you learn are no longer trivial and can no longer be retained by heart just by hearing them, and it goes on throughout a lifetime for many of us. A lot of what makes a good engineer or scientist isn't what they can remember, but how good they are at finding the answer, whether that's by referring to your notes from previous problems you've solved, searching for the answer through experiments, or just Googling for it. If you want to retain that knowledge and not waste time when you need to do something that's close enough to something you've touched in the past, having a well organizing notes taking system is crucial. Over the years I've used a lot of tools myself, and I will review what I consider 3 of the best and most popular options here. Apple Notes If you use an iPhone, like many of us do, the default notes app that comes

This year I've helped a lot of clients with their big data projects, and it's likely that anyone that works in DevOps or even as a regular IT person will have to deal with big data in the coming years. Businesses rely more and more on data analytics to make decisions that wouldn't have been possible before. Whether it's an insurance company adjusting rates based on real time car data streaming in, a security company alerting their agents automatically when something suspicious is detected on one of their many surveillance systems, or even a small business trying to gain more insights from web traffic, big data is everywhere. But before you can deal with big data, you need to know some of the common terms being used, what they refer to, and how they typically apply within the enterprise. This will allow you to successfully engage with the different stakeholders and make sure everyone is on the same page, so projects don't over-promise and under-deliver. Types of data

Big data has become a very important part of doing business, but there's one particular type of data that every business has deal with, and that's web site traffic data. Web analytics, as it's commonly known, is crucial in positioning a web site because it shows you where your users came from, what kind of device they use, their browser type, and whether they clicked on an ad to get to your site. This is all important data to know if you want to run a successful online business. In the early days of the web, there used to be a lot of different analytics options, but these days it's safe to say that the vast majority of sites use Google Analytics. This is because it's well known, integrates with other Google products, and is supported out of the box in many web site builders, making it easy to implement. However, I think it has two big flaws. The first is that it's difficult to get actionable data from it. Over the years, many additional features were added, but

One of the many tasks that a cloud consultant may have to do is access a client's cloud environment. Whether you're having someone create some resources in your AWS network, or hiring an auditor to review your security posture, you need to grant them temporary access in order to allow them to do their job, while restricting what they can do based on the type of work they were hired to do. This means you shouldn't trust them with the root account. Instead, let's see how you can do this securely within just a few minutes. Creating an IAM user In order to login to the AWS console, they will need a username and password. To create one, log into your administrator account, click on Services at the top, and use the search function to go to the IAM page:   There, click on Users on the left side, and then the Add user button: On the first page, you will have to give your new user a name. Make sure to select something that will be obvious who this user is. Then, you have to s

Technology is at the center of most businesses these days, and when an entrepreneur launches a startup, the need to hire a software developer, full stack engineer, or other type of similar employee as part of the team seems like a natural thing. But the Operations side, or what is commonly referred to as IT, is a less obvious prospect. After all, these days anyone can go to one of many hosting providers and click a few buttons, and they'll have a web site. If you don't have a physical office, then that means no need for IT, right? So what is exactly involved in the Operations side of things, and why is it a crucial part of any business? Security When you deploy an infrastructure in the cloud, you have to understand the separation of roles. The cloud provider, whether you're talking about a platform like AWS which allows you to create entire VMs, or a site like WordPress which hosts your web site directly, will only handle their own side of the infrastructure. This includes

  Every business is now a technology company in some way, and just like computers made it into the office a few decades ago, now the cloud is becoming a staple for almost every business. Having worked in many, many client environments, the move to the cloud typically always happens the same way. A department or project manager decides to launch their cloud adoption, which leads to an Azure or AWS tenant being created. Some instances are provisioned, network routes setup, and the deployment grows organically from there. The result of such organic growth tends to be disorganization. Naming conventions are brought in later on, tools are decided on the fly, and even if you use infrastructure-as-code, I've seen many cases where one group is using Terraform, while another is using CloudFormation, and of course there's the break-fix instances where people go into the web console and change things manually. Almost no company older than a couple of years old out there doesn't have

Artificial Intelligence, or AI, is a popular buzzword in tech. There are AI firms all over the world. For most people who aren't in the field of AI, it can seem complicated and mysterious, but in fact it really isn't that hard to understand. While you may think AI is out of reach from anyone without a deep math and computer sciences background, you may be surprised to learn that most applications of AI these days don't have anything to do with complex research. In this post I'll go over what AI is, some examples of what it can do, and how you can get started in the field yourself, should you wish to. What is AI? Looking at the definition, AI is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals . This is a very broad definition, and as you might expect it includes a lot of different technologies. There are many ways to make computers smarter, and many applications where this can be useful. From text to voice to picture

A while back I wrote about how the "work anywhere" culture is taking hold, and that working remotely can't be the exception anymore. People have learned what remote work can look like, and even if it doesn't mean working from home, the concept of working from anywhere is extremely powerful, allowing people to work from an office, a coffee shop, the beach, home, and so on, based on their daily requirements. It brings a lot of fundamental shifts to the way business is done, from no longer needing to commute to a large city every day, to being able to choose your living arrangements that aren't directly tied to your 9-5 job. Another shift we're seeing when it comes to the future of work however is that we're moving away from the traditional employment model to a much more fluid, dynamic setting. A few decades ago, it was expected that someone would become employed by a corporation, work 9-5 every day, and stay there for years if not their whole life. Now eve

As any IT worker would tell you, having proper backups is crucial in any environment. Whether they're done on local hard disks, tape drives or remote locations, having a proper backup solution that includes incremental saves is very important so you can restore data at any point in time, as needed. The same is true for cloud resources, and many people forget that it's not Amazon, Microsoft or Google's role to make sure you don't lose data. Even though a cloud instance you run in AWS is far less likely to lose data than a local VM, thanks to the cloud's natural robustness and all the systems in place to ensure high availability, the chance for a critical hardware failure still exists. There's also room for user error or bad actors coming in and erasing data. If you use AWS, backups can be handled in a very easy way thanks to a service called AWS Backup . It's one of the many features offered by Amazon, and it allows you to schedule backups according to one or

  AWS Elastic Beanstalk is a very popular cloud service that allows you to create an environment for your web app to run in. For example, let's say you have a NodeJS app, and you want to deploy it to the cloud. You could manually create a virtual network, security groups, and then instances for dev, test and production. Or, you could simply have 3 Beanstalk environments. It simplifies the infrastructure, because you tell the service what kind of web app you're going to be running, and the environment is going to include all the prerequisites. Also, updating the app is very easy to automate. This is what we're going to look at here. Before you start, we'll assume you already created your Beanstalk environment. If you didn't, go to the AWS Console and search for Beanstalk. There, create an environment and an application name, making sure you pick NodeJS as the platform, and write down both the application and environment names since we'll need them. By default, th

Starting a business is an exciting time. It was for me, and I'm sure anyone who has an idea and decides to go forward with it suddenly finds themselves very, very busy. Between taking care of business registrations, licensing, taxes, finding the right accountant, lawyer, team members, office space, equipment, marketing and so on. With that in mind, it might not be too surprising to realize that many startups don't take care of their online presence as much as they should. They want to focus on their own businesses, not tech stuff. Here, I'll list a few basic things that any entrepreneur should take care of when starting a business, or at least find someone to help them do it. A professional web site This first one might be pretty obvious, but any business in this day and age needs a web site. Gone are the days of the yellow pages, and if someone can't refer to your site in order to find out what it is you do, then you have major problems. But what kind of site you need

DevOps is a very broad term that can apply to many things. In a sentence, it's a way to tie development and operations together through an automated workflow, allowing builds to quickly and seamlessly go from commit, to test, to production. But DevOps is also the name of a service that Microsoft offers, called Azure DevOps. This service is available both for on-premise using DevOps Server, and in the cloud with DevOps Service. Here, we'll look at deploying a very simple Python web app from GitHub to Azure, first to a test environment and then to production. The first thing you need to do is make sure you have an Azure subscription and go to the Azure DevOps web page to get started. Once on your dashboard, create a new project, name it, and keep the default options: Once the project is created, you'll see links to various features like boards, repos, tests and pipelines. Azure DevOps includes a lot of different services that can help you improve the workflow of your applica

Maybe you're writing a bash script and you want immediate feedback to reach you. Maybe you have a server health check and you need to have SMS alerts go out. Or maybe you just want to be able to send SMS to yourself or your opt-in friends from the Linux command line like a pro. Either way, this little tutorial will show you how you can use Amazon SNS and the AWS command line client to send SMS messages using a single command. AWS configuration Simple Notification Service (SNS) is an AWS service that can be used to send emails or SMS messages to a distribution list. In order to do this, you have a number of topics to which people can subscribe. Then, anything published to these topics will be received by everyone who subscribed to that topic. The first step is to create an SNS topic. You'll need an AWS account, then log into the web console and go to the SNS page. There, you can click on the Topics section and create a new topic: Once done, you will be brought to the details pag

Email used to be the field of the IT experts. The choice you had was setting up your own Microsoft Exchange server on Windows, or a Postfix server on Linux, then deal with everything from DKIM and SPF DNS entries, reputation management, blacklists, spam, and so on. Now, any entrepreneur can go to one of many online services and start their own business email account for very little money, and almost no technical know-how. Hosted email has become the norm, where very few people bother running their own email server anymore. So when I started Dendory Capital last year, I decided to do what most of my clients are already doing, and go for hosted email as well. In fact, I tried three different options, and I'm going to review them here for you today: Google Apps Microsoft Office 365 Amazon WorkMail Now admittedly, our needs are very basic. We're not a large firm, in fact it's just me and a few freelancers. I also wanted to look at this solely from the point of view of email, an

There are a lot of monitoring tools out there, everything from log aggregators, probes, dashboards, graphs, alerting systems, and so on. To make things easier, we can usually divide monitoring into two areas: parsing logs, using something like the ELK stack, and probing systems for specific metrics, which is often done with a tool called Nagios. Both of those tools are great, and I've used them among many others in the past, but one thing I've noticed is how complex they tend to get. Deploying Nagios usually doesn't involve just installing the Nagios dashboard. You need NRPE probes, a backend database, Nconf, and agents on individual systems. Similarly, log collection only goes so far. As soon as you have a couple of servers and applications to monitor, you could be looking at gigabytes of logs coming in every day, so that's why you need to not only collect the logs, but analyze them, parse them, filter them, create graphs, alerts, and so on. My experience using these t