Skip to main content

Posts

Showing posts from 2021

The concept of Minimum Viable Product (MVP) in the cloud

The concept of a Minimum Viable Product (MVP) is a fairly common concept in business. The whole idea is that you want to build fast, ship as soon as you have a working prototype, and then iterate quickly as you get customer feedback. It greatly reduces your initial ramp up cost, allows you to get to market quicker, and improves the final product since you get feedback earlier in the process. This type of methodology is used in every industry, both for physical products and for digital ones such as web apps and software development. The hard part tends to be defining what your MVP will be. You want your product or service to be fully functional, meaning that your users will be able to do anything that was advertised as being a core functionality of the item, but with none of the additional bells and whistles that you may add over time. As expected, this concept also works within the cloud. Whether you're working on new software, a web site, or anything else that runs digitally, it&#

The risks of physical assets vs virtual assets

In this post I wanted to explore some of the risks that people and businesses who invest in virtual assets face, and how this compares with what traditionally is thought more as physical assets. As the world moves more into the digital age, virtual assets are becoming more common and more popular, and with good reason. Understanding the risks is therefore very important. Before I cover the 5 types of risks and how they apply, let's first make sure we all understand what are physical assets and what are virtual assets. Note that those lists aren't meant to be exhaustive. Physical assets are things that you can hold in your hand, or see in the real world. Value is tied to that object. Here are some popular examples: Real estate - There's no question that for most people, their home will be their most valuable physical asset. Art - Even though the art world is concentrated at the top 1%, it's still a vast and very valuable class of assets. Antique cars - Car buffs that a

The long term effect of working from home

  I remember when I still worked as a support worker in an IT department, doing the typical 9 to 5 in a cubicle. One of my dreams was being able to work from a home office, or at least choose my own work location. Whether that be a coffee shop, the beach, or your own home office, it's hard to argue that the prospect of choosing your own working environment is better than being forced into a cubicle for 8 hours a day. While the pandemic has had devastating results all around the world, the silver lining is that it also sped up the adoption of the work from home, or as some as calling it, "work anywhere" culture by a decade or more. Many companies who refused to consider remote work opportunities in the past are now forced to adapt, and this has been an ongoing situation for over a year, with no end in sight. But what are some of the long term impacts of working from home? Here I'll take a look at some tech-related things that have changed and will continue to change go

The best way to store metadata

Big data is all the rage, and as companies scramble to gather as much data as they possibly can, the eventual outcome is typically that you end up with more data than you know what to do with. So in this lake filled with files and objects, it's very easy for data to get lost. This is where metadata comes into play. In a simple way, metadata is additional information about your data, both automatically and manually created. For example, in the case of a picture, the automatic metadata can include the exact time the picture was taken, the type of camera that was used, the GPS location and the resolution, whereas some of the manual metadata can include the name of the author and a description. Having this type of metadata attached to every piece of data you store is crucial, especially as you scale up. One challenge that often presents itself is how to store this metadata, since most file formats don't allow you to add this metadata to the file itself. Using a database One common

Use case: Migrating an on-premise data lake to the cloud

  In this new series of blog posts called Use Cases , I'm going to go over specific projects that we've worked on with clients to address specific needs, how the project went, and the lessons retained from that experience. I won't go into specifics, but I'll cover the important bits. Hopefully this will be helpful if your organization is looking at undertaking a similar project. This week, I'll talk about a project where we had to migrate a large data lake from an on-premise Hadoop infrastructure to the cloud on AWS. Before any project can start, an analysis of the business needs and a design has to be constructed in order to decide what the solution will be. In this particular case, we weren't involved in the initial decision process, but the solution architect came up with a pretty good design. The company had around 30 TB of data sitting in an on-premise data lake. Data engineers and developers would use Hadoop clusters in order to run their ETL pipelines, an

Do you know where your NFT is stored?

  NFTs, or Non-Fungible Tokens , have become the hottest thing in crypto this year. You may have heard in the news about a digital art piece from the artist Beeple selling as an NFT for $69 million, or how the NBA has partnered with a company called Top Shot to sell digital card packs to people. The short definition is that an NFT is a special crypto token that proves ownership of a digital asset. It usually lives on the Ethereum blockchain, although other blockchains also support NFTs, and unlike a traditional real world item, ownership can be proven with complete certainty since the entire transaction is public and stored on the blockchain ledger. So whoever paid those crazy prices for these early releases can be safe to know that their investment will live for as long as the blockchain does, even though it's debatable whether they would ever find a buyer to pay more than they did. But one question that many people aren't taking into account is where these files actually li

When should you automate your work?

Repetition is the key to success in life, whether we're talking about exercising, doing crossword puzzles or other types of professional work, doing the same thing multiple times builds muscle memory. There certainly is a benefit to repeating the same task, whether you're trying to actually build muscles, or in the case of intellectual challenges, if you're trying to learn or improve your mental capacity. But there are tasks that don't really bring any benefit, they just take time. Those are mostly when you spend a lot of time to do menial tasks, things that don't really bring you any benefit. Whether you're doing IT tasks by clicking through UIs, typing commands in your computer, filling numbers in a spreadsheet, writing a report, or classifying papers, these tasks don't tend to bring you any benefit. This is when you should think about automating your work. As someone with a tech background, automation is something I default to very quickly. The good thing

Using the Binance Smart Chain with Python

For most people, crypto is just a way to invest in a highly speculative asset. Bitcoin is seen as digital gold, as something you can buy, hold and sell at a higher price. But crypto currencies and blockchain technologies are used in many more applications, and creating innovative products on top of these networks starts with connecting to the network. Whether you're trying to create a digital wallet, interact with smart contracts, or build a completely new application, you need to use code and connect to the blockchain of your choice. In this post, we'll use Python and connect to the Binance Smart Chain. While there is a lot of crypto documentation out there for JavaScript, the amount is smaller for Python, so hopefully this post will give you a good place to start. Connecting to the chain First, you'll need to download the web3 library. You can use pip to install it on your system: pip install web3 Once installed, open up a Python interpreter and let's initialize the

Running AWS commands using Jenkins pipelines

Jenkins is a well known tool used in the DevOps world to automate processes. Typically, it's mostly used to build and deploy software as part of a CI/CD process, including cloning code from git, running a build job, running tests and deploying the resulting artifacts. However, Jenkins pipelines can be used to automate anything at all. Because of its built-in credentials manager, multi-node architecture and multitude of plugins, it can be worth it to centralize a lot of the repeated tasks you may have inside of Jenkins, rather than having scripts resting on a server somewhere. In this post we'll see how to use a Jenkins pipeline to connect to AWS and retrieve a list of S3 buckets available. Storing credentials The first thing you should do is securely store your credentials. One of the oldest security issue that IT people have had to deal with is the management of credentials. If you use a shell script to run your command, then you need credentials to be stored inside the script

Crypto coins fundamentals

  As part of our technology services here at Dendory Capital, we touch a lot of exciting new industries. One of them is Cryptocurrencies. By now most people know about Bitcoin and how it's been going on a massive growth in the past year, but Bitcoin is just one of many different crypto coins. The landscape is much more vast than it used to be when Bitcoin was the only coin in town, and perhaps the most interesting part of crypto isn't so much the price action but instead all of the technologies behind these coins. In this post I will describe some of the basic fundamentals that drive crypto including blockchain networks, smart contracts and gas fees. The basics of blockchain All crypto currencies work on top of something called the blockchain. In simplified terms, a blockchain is a chain of blocks, linked together in a secure manner, and published on a ledger. This is how crypto coins are created, and how transactions are recorded. For example, let's say you have one Bitcoi