Mapping the internet by botnet
I have always been intrigued by unique datasets. Something about the varying levels of complexity and the numerous inferences that can be drawn from seemingly random collections of information has always been intriguing to me. In this case, it is no different. We know it as the sum of decentralized connections between interconnected computers communicating in roughly the same language. The layperson knows it as the internet. It turns out that a researcher has been able to compile a visually stunning map of the internet as we know it. Through a series of relatively simple methods, and egregiously lax security settings on target devices, he was able to map out approximately 420,000 nodes by sending and receiving ping requests. His research walks a fine line between hacking and mapping.
In his writing he notes,”we discovered an amazing number of open embedded devices on the Internet. Many of them are based on Linux and allow login to standard BusyBox with empty or default credentials. We used these devices to build a distributed port scanner to scan all IPv4 addresses. These scans include service probes for the most common ports, ICMP ping, reverse DNS and SYN scans. We analyzed some of the data to get an estimation of the IP address usage”. This speaks to the shockingly insecure state of many many devices connected to the network we know and love. Regardless of the insecurity, the researchers work makes for some fascinating imagery. Most notable to me are the high concentrations in eastern China and the severe lack of activity in Africa. There can be many conclusions drawn from the state of the developing world as a whole.
While the map is still limited to IPV4 and does not take IPV6 into consideration, it is still quite a sight to behold. Additionally, what is represented is a wide variety of linux based systems with a predefined threshold of processing power. While some may argue that this is a serious security breach, it would be in the best interest of the researcher to stay anonymous in spite of the procedures he used to refrain from any malicious intent. I agree with the steps utilized to notify the owners that their device had been accessed and even the email contact information. Some parties on the other hand may not feel the same. I firmly believe that any admin who has the audacity to connect a device and leave its default administrative credentials as root/root should have his or her “internet card” revoked so to speak. Sadly, a brief google query string can turn up open FTP sites, millions of freely accessible web connected printers and a shocking number of unprotected webcams.
As the world becomes more and more interconnected, we will continue to see graphs like this only increase in density and complexity. With something as dynamic as the internet we know today, illustrations such as this are roughly outdated the moment they are published. The developing world in conjunction with ubiquitous mobile platforms will further extend the exponential growth of this world over wire.
Link to research paper: http://internetcensus2012.bitbucket.org/paper.html
One half second of High Frequency Trading
In the business of high frequency trading, milliseconds can mean the difference between sizable gain or catastrophic loss. For those who are unaware as to what is really going on in today’s financial markets, the following video will provide a bit of insight to a world where speed is king.
This video illustrates just one half second of trading by high frequency systems. The boxes represent exchanges and each “particle” is a quote change. Again, just to reiterate, the video has been slowed down to allow us to appreciate the speed of these transactions.The time at the bottom of the screen is Eastern Time HH:MM:SS:mmm (mmm = millisecond). A millisecond (ms) is 1/1000th of a second.
High speed trading and the development of quantitative algorithmic trading on wall street is highly secretive and still essentially misunderstood by the general public. In recent years the industry has exploded and been responsible for a massive migration of some of the smartest minds in physics, math, and computer science. This is the future of our financial system. Trillions of dollars reliant upon the successful operation of servers in datacenters strewn all over the world. We are simply the programmers behind the machines manipulating code and assessing profitability.
As we develop these systems to handle trades and execute orders, technology will change, algorithms will develop and milliseconds will give way to nanoseconds. It is my fear that the recent “flash crashes” due to out of control algos only represent the beginning as we tip toe on the precipice of global financial catastrophe.
…Do you hear that? It’s the Technological Singularity inching it’s way closer.
video credit: Eric Hunsader
Integer overflow and what it means for the year 2038
Most of us can remember the panic and “rush to patch” era of the late 1990′s. More specifically, new years day 2000. Millions of lines of code were inspected in an effort to remove any improperly formatted date strings to reflect the year 2000. Incorrectly formatted strings would read the year 1900 when the date rolled over. True to form, the issue even had its own cool nickname, Y2K problem, the Millennium bug, the Y2K bug, or simply Y2K. Having your bank records negatively affected by an improperly formatted date string cannot be too enjoyable.
Today we have a new situation that may warrant some attention. It is called the 2038 problem. While still a quarter century away, it should shed light on formatting and proper structure for future software. The issue at hand is that any software and systems that both store system time as a signed 32-bit integer, will reach maximum binary storage at 03:14:07 UTC on Tuesday, 19 January 2038. After this point, the time data will then be stored as a negative number counting up to 0. That date will then represent December 1901 instead of january 19, 2038. The root cause of this problem is a result of integer overflow.
Integer overflow occurs when a math operation creates a value too large to be stored within the available storage space. Think of the odometer on your car. It has a maximum value. If you arrive at that value and add 1, the entire string will then be returned to 0. Integer overflow is not to be confused with buffer overflow as it does not cause memory safety violations.
So what does this mean going forward? While there is no universal solution to the problem it is important to recognize the rigidity of storing date values in binary data files. 64bit systems utilize signed 64-bit time_t integers. This pushes the maximum date, or the end of time, to over 292 billion years in the future. It is the upper boundary of current 32 bit software and systems that will need modification. We have 25 years to find a solution unless your system is reflecting future dates, then you should be prepared to adjust for the change. Oh yes, and don’t forget about leap seconds!
What a DDOS attack looks like
If you have ever wondered what a Distributed Denial of Service (DDOS) attack looks like, this video will give you some insight.
Just imagine hundreds of thousands of seemingly random IP addresses bombarding a handful of IP’s with external requests. On the other end of this data stream could be and has been, your bank, favorite online retailer and even the US government. Some of the more recent attacks have seen packet per second rates come close to 500k. The more poorly secured computers and vulnerable network connected devices get infected, the greater these numbers become.
You never know, your machine may be compromised right this moment. It could be sitting infected, dormant, just waiting until it receives its instructions to join the bot-net of other infected devices. It may sound like a stretch but this is exactly how these attacks achieve their levels of effectiveness. A testament to the power of exponents.
Cloud based development environment
As a Computer Science student i knew that the current tools we utilized in class would undergo a radical transformation. If history is to repeat itself , we would see some amazing outcomes i argued. Such has been the case in the development landscape over the last few years.
A nice highlight is the advent of web 2.0 and cloud computing. These tools allow for greater colaboration and development flexibility. A tool that takes full advantage of this is www.coderun.com/
From site: “CodeRun Studio is a cross-platform Integrated Development Environment (IDE), designed for the cloud. It enables you to easily develop, debug and deploy web applications using your browser. CodeRun Studio can be used instead or alongside your existing desktop IDE. You can upload existing code in order to test it in the cloud or for sharing with your peers.”
I will definitely dust off the coding skills and run some tutorials through this resource.
Emergency DNS list
Here is the listing of sites and their ip addresses. Useful during DNS outages/takedowns.
# Social media
# Torrent sites
# Social networking
# Live Streaming Content
# File Sharing