There’s so much data being generated in today’s world that we need platforms and frameworks that it’s mind boggling. This field of study is called Big Data Analysis. With so much data lying around, often ranging in petabytes and exabytes, we need super powerful systems to process it. Not only that, we need to do it high efficiency. If you try to do it using your regular ways, you will never be able to do anything in time, let alone doing it in real-time. This is where Apache Spark comes into picture. It is an open source big data processing framework that can process massive amounts of data at high speed using cluster computing. Let’s see how we can install it on Ubuntu. Continue reading “How To Install Apache Spark On Ubuntu”
Tag: Parallel Processing
Using Multiple CPU Cores With Command Line Tools
All of you must have heard about how the processors in our laptops have multiple cores. It’s good that the technology is advancing in that direction. When people write programs, they can utilize these cores to increase the speed of computation. But most of the inbuilt commands don’t use these cores unless specified explicitly. If you ever want to add up a very large list, say hundreds of megabytes, or just look through it to find some particular value, you would write a simple program to do it. But going through so much data takes a lot of time if you just use a single thread. The same is true for tools like grep, bzip2, wc, awk, sed, etc. If the last sentence looked like jibber-jabber, then you should probably google those things before you proceed. They are singly-threaded and will just use one CPU core. So how do we use multiple cores in these situations? Continue reading “Using Multiple CPU Cores With Command Line Tools”
Quantum Computer
We are always hungry for more processing power. Whenever we buy computers, we always look for the ones with higher configuration. The computer manufacturers thrive to give us more speed and processing power at lower costs so that they can differentiate themselves from their competitors. With the emergence of so many technologies, our need for processing power is increasing by the day. Moore’s law states that the number of transistors on a microprocessor will double every 18 months. This law has been more or less true up until now but it is getting saturated. So we need to look into a different direction if we want to keep up. Quantum computers can perform calculations significantly faster than silicon-based computer. So what are quantum computers? How can they perform calculations so quickly? Continue reading “Quantum Computer”
Perpetual Enigma 