Higher-Order Functions with Spark 3.1

Complex data structures, such as arrays, structs, and maps are very common in big data processing, especially in Spark. The situation occurs each time we want to represent in one column more than a…

Better Than GPT-3 — Meet BlenderBot 2.0: Facebook’s Latest Chatbot

BlenderBot 2.0 is the latest conversational AI from the Facebook AI team.

How to compare 2 dataframes easily

4 years ago, I was involved in a data migration project and was part of a business department within a global financial institution. The business department needed to complete the reconciliation…

towardsdatascience.com 10 hours ago

AlphaFold-based databases and fully-fledged, easy-to-use AlphaFold interfaces poised to revolutionize biology

Not only computational but also experimental biology. Thoughts on the future of data science niches in biology. In a recent story I covered the release of the academic paper describing AlphaFold’s…

towardsdatascience.com 10 hours ago

Data to Model to API: An End-to-End Approach

In this blog, we are going to go through the entire journey of a machine learning-based use case. We are going to start by data processing, move onto building a data pipeline to feed to model for…

towardsdatascience.com 11 hours ago

What in the World is Going on with Data Catalogs?

It seems like every time I refresh my Twitter feed, a new startup launches “the world’s greatest data catalog ever.” And that’s exciting! If a company is able to build the next best catalog since…

towardsdatascience.com 13 hours ago

5 Data Transformers to know from Scikit-Learn

As Data scientists, we are often faced with many situations where we faced difficulty when exploring data and developing machine learning. The struggle could come from the statistical assumption who…

towardsdatascience.com 13 hours ago

Popular Interview Question that Reduced me to SDE-1 from SDE-2 Role

“Imagine this scenario” the interviewer started after failing to completely corner me with his repository of questions he came prepared with. This man here held in his hand one of my career…

towardsdatascience.com 13 hours ago

The Ultimate Guide to Emotion Recognition from Facial Expressions using Python

Emotion is one of the very few words in the English language that do not have a concrete definition and it is understandable. It is abstract. Yet, almost every decision we have ever made in…

towardsdatascience.com 13 hours ago

10 Best SQL Editor Tools in the Market

In modern computing environments, diversified database platforms are the norm. Over the years, the demands of effectively using enterprise data resources have made it practically impossible for…

towardsdatascience.com 13 hours ago