Data Science Using Bare-Metal Servers?

The 5 arguments for “beyond the cloud”

There are 5 cases in which you may want to go bare(-metal) with your mission-critical data projects

Data Science on the cloud?

Cloud computing services are the future of data science, according to just about everyone. To suggest otherwise is practically blasphemy these days — and yet this mindset comes with some serious downsides. In fact, for start-ups and small companies, Cloud servers can be your ultimate downfall.

When I started my company Evo, I was all-in on the Cloud. After all, some of the same advances in computing that enabled Cloud services at a mass scale made Evo’s prescriptive AI possible. Leveraging Big Data in new ways is critical for transformative impact. Furthermore, we received free credits from Azure and AWS, which we love(d).

But as Evo scaled, the costs began to spiral out of control. Because of the many resources required by the many IO operations carried out by Evo data engineers, data scientists and deployed products, it cost Evo tens of thousands of dollars a month just for basic server needs.

Why bare-metal with Serverplan

Of course, simply finding out that Cloud servers for our data science projects were expensive didn’t mean they weren’t worth the costs. We are willing to invest in the best infrastructure to serve our clients. But Cloud wasn’t just costly. It was also inconvenient.

Cloud servers are primarily provided by three massive tech companies that can dictate terms with little regard for the needs of small companies like ours. Unless you can afford to pay a considerable premium, you are usually unable to afford cutting edge technology. Moreover, without services replication, which is not always feasible due to costs and technological requirements, your servers can restart mid-analysis, jeopardising hours of elaboration (which you already paid for!). In response, our tech team made a bold suggestion: to look at investing in bare-metal servers instead.

While we toyed with building all our own servers in house, we quickly decided that would be another high cost, both in terms of staff, time, and other resources. We wanted to regain control of our data relatively quickly, and the scale-up would be too slow. So instead, we looked for bare-metal server providers. That’s when we found Serverplan.

Serverplan was the perfectly balanced service provider for our needs. After a lengthy vendor comparison, we saw that they were best able to work with us like a partner rather than an anonymous hosting company: one we could trust to provide consistent, high-quality service. We were also impressed by the regularity with which they actively monitor and upgrade their hardware, ensuring minimal risks in terms of degradation, availability and other threats to our data. Most importantly, these bare-metal servers would be ours to manage with consistent, predictable costs over time. We would save thousands every month with the same capacity and the ability to scale with new clients simply by adding new servers.

We worked with Serverplan to make the switch and have since been happy with their service. While we keep some Cloud services for specific client delivery needs and as backups, bare-metal servers have allowed Evo to scale more efficiently, even as a company dependent on Big Data. For our particular data science projects, bare-metal is the right fit.

When should I go bare(-metal)?

So now what? If Serverplan was a better partner for us than the Cloud providers, how do you know if bare-metal servers are right for you? Five key considerations may make bare-metal servers more appealing.

1. Your project needs to process a lot of data regularly, and you know when you need those resources in advance.

Cloud is most cost-effective when you have an elastic workload. The best thing about using Cloud servers is the scalability; you can go from 0 to 100 and back to 10 quickly. However, in our case, we get a lot of data from our clients on a daily basis that needs to be added to data that has already been ingested. We use that data for scheduled and sometimes manual analyses that almost always require resources in predictable timelines but can almost never scale down.

Bare-metal may be less responsive in terms of scale-up, but it allows you to deal with massive amounts of input/output data at a lower cost. You can invest in a long-term, predictable solution that is always available rather than simply paying for an ephemeral Cloud solution that you will need to spin up again and again regularly.

After all, when you run on Cloud, you don’t just pay for disk storage. You also are paying for other resources like input/output operations, capped to the disk tier and VM limits you initially set. You cannot burst or have a high throughput unless you pay in advance for premium resources, incurring costs regardless of actual use: a clear example being Azure pre-allocated Storage. This is quite expensive and, quite frankly, unnecessary when you can estimate your data and throughput needs in advance.

2. You have large amounts of client data that you need to know where and how it is secured.

Data privacy is critical. Clients cannot afford leaks, and we certainly do not want anything to risk their trust or data security. While most Cloud providers have become more transparent about where and how they store and secure data since the passage of the GDRP, there is still no way to be 100% sure what is happening with your client data on the Cloud.

Bare-metal servers require you to invest in proper security protocols. Still, you have more control of the data, making most clients feel more comfortable sharing such critical information. If your data science project requires a high level of data security, it is easier to ensure that security when you have more control. Bare-metal allows for that.

3. You have the talent to run and maintain a server and cloud infrastructure in-house.

Someone is running and maintaining your server at all times on the Cloud. But even when it’s on the Cloud, the same people on your team have to monitor your services and keep them healthy.

On the one hand, a major benefit of the Cloud is that you don’t have to spend the internal resources on deploying the operative systems or configuring a virtualised infrastructure. You know that the technical staff on hand has plenty of practice in these operations and should handle any problems that emerge.

On the other hand, you deprive your own team of the opportunity to develop and deepen new skill sets. Suppose you already have data engineers and infrastructure experts with this knowledge who are simply overseeing the Cloud operations provided externally. In that case, you may find that they can better serve your needs with the hands-on approach of bare metal. It would be almost impossible to provide high-quality client service using a bare-metal server without the proper internal resources. However, if you already have them in-house, like at Evo, it can be a great way to challenge and fully employ your team.

4. You need to control when the server restarts or experiences maintenance because of the immense impact such a disruption could have on operations.

Server maintenance and restarts are inevitable. You may need to be offline for just a few minutes, or it could be even hours of maintenance work. When this type of work happens on the Cloud, they often shut off your server with or without warning. In fact, most Cloud contracts require you to consent to this unpredictability. It’s a non-negotiable hassle of the Cloud whose only workaround is to double the cost by deploying the same resources over high availability groups — and even this approach might not be fully supported by the types of services you are deploying.

When you are in the middle of a critical operation that depends on that server, this could undo hours of work and severely impact operations — and clients. When you use bare-metal servers, you can schedule server maintenance for times that will have minimal impact on your projects. Even when incredibly rare emergency restarts are unavoidable, you can warn affected teams and better mitigate the impact.

Bare-metal puts you in control of disruptions, allowing you to minimize the impact altogether.

5. You are a start-up or a small business that could not survive without your servers for more than a day.

For a tech company like Evo, our servers are absolutely critical. None of our tools or infrastructure can run without them. It would be devastating if we were to lose servers and data for even one day.

Big Cloud providers do not need our relatively small business. If a processing issue delayed payment, a change in terms eliminated our product, or some other decision otherwise cut our access to these servers, these Cloud providers would have little motivation to work with us to prevent issues. Evo, on the other hand, would be in dire straits right away.

By running our bare-metal servers, we do not risk losing our data or halting our operations due to an unforeseen change in Cloud business. If your data science project or business is similarly dependent on your servers, you should think twice before putting all your faith in the Cloud.

Reconsidering your options

No one would say that bare metal servers are always best. It would be an absurd statement. Evo still uses a hybrid solution that depends on the Cloud for certain operations. But bare-metal servers can be an excellent solution for specific needs.

At Evo, the level of control we maintain by running our own servers is beneficial, especially since the cost is so much lower. We have a supportive partner in Serverplan as we return to bare-metal servers. Cloud is an incredible technology, but it isn’t the only way forward in the future.

About the author