Top 10 Big Data Technologies Of Present Times

Over last few years Big Data technologies are getting due attention and there are several trends and innovations in this space in recent times.    Wednesday, October 22, 2014:  Big Data is a concept which is quite broad and comprises several trends and technology developments. Over last few years Big Data technologies are getting due attention and there are several trends and innovations in this space in recent times. Here we'll discuss top ten emerging Big Data technologies. 1. Column-oriented databases:  Traditional databases are excellent in online transaction processing but when it comes to query performance while data volumes grow, these databases fall short on performance. The new column-oriented databases store data and focuses on columns and not rows. It allows huge data compression and faster query times. 2. Streaming Big Data analytics:  There are several projects in this section including Storm, Spark, Data Torrent, Spring XD and SQL Stream. Apache Storm is an open source distributed real-time computation system which simplifies streams of data and real-time processing. Spark is a data processing platform which is compatible with Hadoop. DataTorrent is a real-time streaming platform which enables businesses to perform data processing. Spring XD supports streams for event driven data while SQLStream provides a distributed stream processing platform for streaming analytics, visualization and continuous integration of machine data. 3. Schema-less databases, or NoSQL databases:  This database category includes key-value stores and document stores. This database focuses on storage and retrieval of large volumes of unstructured, semi-structured or even structured data. 4. SQL-in-Hadoop:  This technology includes Apache Hive, Shark, Apache Drill, Presto and Phoenix among many others. It helps in making queries and it also manages large datasets in distributed storage. Shark is a data warehouse system which supports Hive's query language. Apache Drill is an Apache incubation project and it's designed for scalability. It's backed by MapR. Presto is an open source distributed SQL query engine and Phoenix is an open source SQL query engine for Apache Hbase. 5. MapReduce:  It's a programming paradigm which allows massive job execution scalability against thousands of servers or clusters of servers. Its two tasks are Map task and Reduce task. It converts any input dataset into different set of value pairs while reducing set of tuples. 6. Hadoop:  Hadoop is an open source platform for handling Big Data which can work with multiple data sources. It has other applications too and it's largely used for changing data like location-based data from weather or traffic sensors, web-based or social media data or machine-to-machine transactional data. 7. PIG:  PIG brings the Hadoop project close to developers and business users and it's used by Perl like language allowing query execution over data stored on a Hadoop cluster. PIG was a project by Yahoo! But now it's completely open source. 8. Big Data Lambda Architecture:  Lambda Architecture is a hybrid platform which combines real-time data and pre-computed data to provide a near-real time view of the data at all times. Its frameworks include Summingbird by Twitter and Lambdoop. 9. PLATFORA:  It almost copies Hadoop and it requires developer knowledge to operate. It's a platform which turns queries into Hadoop jobs with immediate effect and creates an abstraction layer to simplify the datasets in Hadoop. 10. SkyTree:  It's a high performance machine learning data analytics platform which handles Big Data. It's an essential part of Big Data. Courtesy: TechRepublic and InfoQ  Sanchari Banerjee, EFYTIMES News Network

10 Programming Languages You Should Learn in 2014

by Rebecca Hiscott

The tech sector is booming. If you've used a smartphone or logged on to a computer at least once in the last few years, you've probably noticed this.

As a result, coding skills are in high demand, with programming jobs paying significantly more than the average position. Even beyond the tech world, an understanding of at least one programming language makes an impressive addition to any resumé.

SEE ALSO: The 8 Hottest Tech Jobs of 2014

The in-vogue languages vary by employment sector. Financial and enterprise systems need to perform complicated functions and remain highly organized, requiring languages like Java and C#. Media- and design-related webpages and software will require dynamic, versatile and functional languages with minimal code, such as Ruby, PHP, JavaScript and Objective-C.

With some help from Lynda.com, we've compiled a list of 10 of the most sought-after programming languages to get you up to speed.

1. Java

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: Java is a class-based, object-oriented programming language developed by Sun Microsystems in the 1990s. It's one of the most in-demand programming languages, a standard for enterprise software, web-based content, games and mobile apps, as well as the Androidoperating system. Java is designed to work across multiple software platforms, meaning a program written on Mac OS X, for example, could also run on Windows.

Beginners are advised to learn C/C++ first, as Java is not the most user-friendly of languages.

Where to learn it: Udemy, Lynda.com, Oracle.com, LearnJavaOnline.org.

2. C Language

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: A general-purpose, imperative programming language developed in the early '70s, C is the oldest and most widely used language, providing the building blocks for other popular languages, such as C#, Java, JavaScript and Python. C is mostly used for implementing operating systems and embedded applications.

Because it provides the foundation for many other languages, it is advisable to learn C (and C++) before moving on to others.

Where to learn it: Learn-C, Introduction To Programming, Lynda.com, CProgramming.com,Learn C The Hard Way.

3. C++

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: C++ is an intermediate-level language with object-oriented programming features, originally designed to enhance the C language. C++ powers major software like Firefox, Winampand Adobe programs. It's used to develop systems software, application software, high-performance server and client applications and video games.

Where to learn it: Udemy, Lynda.com, CPlusPlus.com, LearnCpp.com, CProgramming.com.

4. C#

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: Pronounced "C-sharp," C# is a multi-paradigm language developed by Microsoft as part of its .NET initiative. Combining principles from C and C++, C# is a general-purpose language used to develop software for Microsoft and Windows platforms.

Where to learn it: Udemy, Lynda.com, Microsoft Virtual Academy, TutorialsPoint.com.

5. Objective-C

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: Objective-C is a general-purpose, object-oriented programming language used by theApple operating system. It powers Apple's OS X and iOS, as well as its APIs, and can be used to create iPhone apps, which has generated a huge demand for this once-outmoded programming language.

Where to learn it: Udemy, Lynda.com, Mac Developer Library, Cocoa Dev Central, Mobile Tuts+.

6. PHP

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: PHP (Hypertext Processor) is a free, server-side scripting language designed for dynamic websites and app development. It can be directly embedded into an HTML source document rather than an external file, which has made it a popular programming language for web developers. PHP powers more than 200 million websites, including Wordpress, Digg andFacebook.

Where to learn it: Udemy, Codecademy, Lynda.com, Treehouse, Zend Developer Zone,PHP.net.

7. Python

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: Python is a high-level, server-side scripting language for websites and mobile apps. It's considered a fairly easy language for beginners due to its readability and compact syntax, meaning developers can use fewer lines of code to express a concept than they would in other languages. It powers the web apps for Instagram, Pinterest and Rdio through its associated web framework, Django, and is used by Google, Yahoo! and NASA.

Where to learn it: Udemy, Codecademy, Lynda.com, LearnPython.org, Python.org.

8. Ruby

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: A dynamic, object-oriented scripting language for developing websites and mobile apps, Ruby was designed to be simple and easy to write. It powers the Ruby on Rails (or Rails) framework, which is used on Scribd, GitHub, Groupon and Shopify. Like Python, Ruby is considered a fairly user-friendly language for beginners.

Where to learn it: Codecademy, Code School, TryRuby.org, RubyMonk.

9. JavaScript

IMAGE: MASHABLE COMPOSITE. IMAGE: WIKIMEDIA COMMONS

What it is: JavaScript is a client and server-side scripting language developed by Netscape that derives much of its syntax from C. It can be used across multiple web browsers and is considered essential for developing interactive or animated web functions. It is also used in game development and writing desktop applications. JavaScript interpreters are embedded in Google's Chrome extensions, Apple's Safari extensions, Adobe Acrobat and Reader, and Adobe's Creative Suite.

Where to learn it: Codecademy, Lynda.com, Code School, Treehouse, Learn-JS.org.

SEE ALSO: Is JavaScript the Future of Programming?

10. SQL

What it is: Structured Query Language (SQL) is a special-purpose language for managing data in relational database management systems. It is most commonly used for its "Query" function, which searches informational databases. SQL was standardized by the American National Standards Institute (ANSI) and the International Organization for Standardization (ISO) in the 1980s.

Where to learn it: Lynda.com, SQLCourse.com, TutorialsPoint.com, SQLZoo.net.

The 3 reasons why Big Data benefits your business

Big Data, Simple Guidance

by Microsoft

MSFT for Work 

6 Jan 2014 12:49 PM 

• 0

Big data is very simple—it’s an extremely large set of data—but the confusing part is knowing what to do with all of that information. Big data can either be a tremendous asset to your business or a time-sapping mire.David McJannet, VP of marketing for Microsoft partner Hortonworks, provides a simplified definition to shed some light on what big data is and why it’s important:
Big data is about building new analytic applications based on new types of data, to better serve your customers and drive a better competitive advantage.
There are three key reasons data insights are important to your business:

1. You can harvest intelligent data by using data solutions that comb the entire web and gather relevant information to create actionable analytics. Getting the right data is the first step to actionable insights.
2. Empower employees with real-time insights by delivering data visualization to anyone in your organization. Data will be much more helpful if your employees can use it easily.
3. Create a connected, data-driven organization that eliminates silos by connecting people through shared processes and data. Open the floodgates and give everyone access to data, and the insights driven by that data.

Two departments within most organizations that will be affected by more effective business insights are Marketing and Finance:

1. Reimagine Marketing driven by deep customer insights to create engaging, personal advertising. Marketing can be more personal than ever before—one of the best ways for it to work effectively.
2. Reimagine Finance by translating data into business impact. Assess and control risk, find ways to reduce cost, and strategize growth and competition. Predictive models will enable your business to anticipate changes most businesses were guessing at 10 years ago.

Big data is about driving actionable insights for your business. The data-driven organization will be able to better understand their customers and business challenges to provide a competitive advantage.

BIG Data

BIG Data: Is this about data at all?


Inspired by: Rafal Lukawiecki’s seminar about Business Analytics and Big Data, Microsoft Norway

Intro

Wikipedia defined BIG Data as a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. But this definition does not include the main reason why Big Data is actual nowadays and what was the purpose to re-invent this technology. Here is a visualization map for BIG Data and how is that large amount of data is generated:

Fig1. Big Data visualization by WIPRO

As you can see from the figure, Big Data aims to represent a large set of data with a single point (a value or a expression) so that make sense for us.

BIG DATA: how big is World data?

Big Data is one of the most famous words around the world describing a new technology that will handle the big amount of data that is generated every day for analytic purposes. But, anyway handling Big Data is not any problem because we are witnessing everyday that hardware capacity expands as data volume expands and also hardware is getting cheaper day by day. So the definition BIG is not at all the case of Big Data, so the hardware capacity can hold whatever BIG Data can be. The average data set of the whole World is calculated to be 1.5 GB and that is an average memory stick, even though an average RAM (in-memory) capacity.
Anyway, if the case is not capacity and size then what is it?

BIG Data: what about data?

Data is important part of BIG Data, but is this meaning of the concept behind the BIG Data? The answer is NO and to be correct BIG Data is just a meaningless buzzword created only for masses. Behind this name does not exist any concept of the Big Data. If you look at the data you can’t say nothing than is big or small, has 1, 2, 3 … n sources and is rapidly/slowly expanding etc…, but that is not what BIG Data is interested to solve at all. If you think the size is the matter, then you are wrong again, so Big Data is not about BIGness at all.

BIG Data is interested to answer the users and not developers, is not an optimizing tool but it is actually an answering machine.

BIG Data: The real case?!

The real deal in BIG Data is that BIG Data tries to generate a single answer (I like to call: the single truth) from a huge input of data. If the answer is the only output of BIG Data processing, so logically BIG Data is dependent on QUESTION. So, the real deal behind BIG Data is the question itself. If you are in a dilemma whether to choose or not BIG Data technology over traditional database technologies you should look not in the size and not in the data itself, but simply in your queries that you are going to use on that set of data. So, I agree totally with Rafal when he says that the reason existence of Big Data technologies are in the answer that we want to get from BIG Data.

Conclusion
BIG Data is just another buzz word without having to deal with the contest itself, but better when we know before we use it. Even thou, we can’t change the trends for this buzzword; at least we can support and use the technology as much as we can.

IT Training By Experts

At Technitrain we believe the best way to learn a technology is to learn from an expert. Our courses are taught by top consultants who have a wealth of hands-on experience to share and who can answer all of your difficult questions. You'll acquire the practical skills you need to do your job as well as learn the tips and tricks that only the experts know.

Here is the link to more information.

Our trainers are the best in their field: Microsoft MVPs, authors and well-known bloggers such as Chris Webb, Gavin Payne, Christian Bolton.

Big Data, the amazing thing

Big data is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target currently ranging from a few dozen terabytes to many petabytes of data in a single data set.

In a 2001 research report^[15] and related conference presentations, then META Group (now Gartner) analyst, Doug Laney, defined data growth challenges (and opportunities) as being three-dimensional, i.e. increasing volume (amount of data), velocity (speed of data in/out), and variety (range of data types, sources). Gartner continues to use this model for describing big data.






Whether through blogs, twitter, or technical articles, you’ve probably heard about Big Data, and a recognition that organizations need to look beyond the traditional databases to achieve the most cost effective storage and processing of extremely large data sets, unstructured data, and/or data that comes in too fast. As the prevalence and importance of such data increases, many organizations are looking at how to leverage technologies such as those in the Apache Hadoop ecosystem. Recognizing one size doesn’t fit all, we began detailing our approach to Big Data at the PASS Summit last October. Microsoft’s goal for Big Data is to provide insights to all users from structured or unstructured data of any size. While very scalable, accommodating, and powerful, most Big Data solutions based on Hadoop require highly trained staff to deploy and manage. In addition, the benefits are limited to few highly technical users who are as comfortable programming their requirements as they are using advanced statistical techniques to extract value. For those of us who have been around the BI industry for a few years, this may sound similar to the early 90s where the benefits of our field were limited to a few within the corporation through the Executive Information Systems.

Analysis on Hadoop for Everyone

Microsoft entered the Business Intelligence industry to enable orders of magnitude more users to make better decisions from applications they use every day. This was the motivation behind being the first DBMS vendor to include an OLAP engine with the release of SQL Server 7.0 OLAP Services that enabled Excel users to ask business questions at the speed of thought. It remained the motivation behind PowerPivot in SQL Server 2008 R2, a self-service BI offering that allowed end users to build their own solutions without dependence on IT, as well as provided IT insights on how data was being consumed within the organization. And, with the release of Power View in SQL Server 2012, that goal will bring the power of rich interactive exploration directly in the hands of every user within an organization.
Enabling end users to merge data stored in a Hadoop deployment with data from other systems or with their own personal data is a natural next step. In fact, we also introduced Hive ODBC driver, currently in Community Technology Preview, at the PASS Summit in October. This driver allows connectivity to Apache Hive, which in turn facilitates querying and managing large datasets residing in distributed storage by exposing them as a data warehouse.

Industry News

Cloud computing changing future of BI

2012-03-23
With the onset of cloud technology, many different sectors of the business and personal world are rapidly changing. From the focus on personal computers to mobile tablets, and from legacy systems to IaaS and SaaS systems, it goes without saying that in the coming years consumers can expect a technological revolution.

All these emerging systems are impacting business intelligence as well. A recent IDC study predicts that the market for big data technology and business intelligence software will grow from $3.2 billion in 2010 to $16.9 billion in 2015.

Furthermore, Gartner predicts that companies will nearly be forced into using these new technologies or risk losing a competitive edge. According to a new Gartner study, 85 percent of Fortune 500 corporations will fail to effectively use big data to get an advantage.

According to Jeff Kaplan, managing director of THINKstrategies, businesses must adopt some semblance of business intelligence and analytics software in order to maintain an edge that may have previously been established with legacy systems.

"Without all these cloud-based resources and tools, most organizations would be unable to cope with today's explosive growth of data," he said.

Predictive Analytics vs Data Mining

Technology Cycle:
Data warehousing is a mature technology, with approximately 70 percent of Forrester Research survey respondents indicating they have one in production. Data mining has endured significant consolidation of products since 2000, in spite of initial high-profile success stories, and has sought shelter in encapsulating its algorithms in the recommendation engines of marketing and campaign management software. Statistical inference has been transformed into predictive modelling. As we shall see, the emerging trend in predictive analytics has been enabled by the convergence of a variety of factors.

Technology Hierarchy:
In the technology hierarchy, data warehousing is generally considered an architecture for data management. Of course, when implemented, a data warehouse is a database providing information about (among many other things) what customers are buying or using which products or services and when and where are they doing so. Data mining is a process for knowledge discovery, primarily relying on generalizations of the "law of large numbers" and the principles of statistics applied to them. Predictive analytics emerges as an application that both builds on and delimits these two predecessor technologies, exploiting large volumes of data and forward-looking inference engines, by definition, providing predictions about diverse domains.

Methods:
The method of data warehousing is structured query language (SQL) and its various extensions. Data mining employs the "law of large numbers" and the principles of statistics and probability that address the issues around decision making in uncertainty. Predictive analytics carries forward the work of the two predecessor domains. Though not a silver bullet, better algorithms in operations research, risk minimization and parallel processing, when combined with hardware improvements and the lessons of usability testing, have resulted in successful new predictive applications emerging in the market. (Again, see Figure 1 on predictive analytics enabling technologies.) Widely diverging domains such as the behaviour of consumers, stocks and bonds, and fraud detection have been attacked with significant success by predictive analytics on a progressively incremental scale and scope. The work of the past decade in building the data warehouse and especially of its closely related techniques, particularly parallel processing, are key enabling factors. Statistical processing has been useful in data preparation, model construction and model validation. However, it is only with predictive analytics that the inference and knowledge are actually encoded into the model that, in turn, is encapsulated in a business application.

Definition
This results in the following definition of predictive analytics: Methods of directed and undirected knowledge discovery, relying on statistical algorithms, neural networks and optimization research to prescribe (recommend) and predict (future) actions based on discovering, verifying and applying patterns in data to predict the behavior of customers, products, services, market dynamics and other critical business transactions. In general, tools in predictive analytics employ methods to identify and relate independent and dependent variables - the independent variable being "responsible for" the dependent one and the way in which the variables "relate," providing a pattern and a model for the behavior of the downstream variables.

In data warehousing, the analyst asks a question of the data set with a predefined set of conditions and qualifications, and a known output structure. The traditional data cube addresses: What customers are buying or using which product or service and when and where are they doing so? Typically, the question is represented in a piece of SQL against a relational database. The business insight needed to craft the question to be answered by the data warehouse remains hidden in a black box - the analyst's head. Data mining gives us tools with which to engage in question formulation based primarily on the "law of large numbers" of classic statistics. Predictive analytics have introduced decision trees, neural networks and other pattern-matching algorithms constrained by data percolation. It is true that in doing so, technologies such as neural networks have themselves become a black box. However, neural networks and related technologies have enabled significant progress in automating, formulating and answering questions not previously envisioned. In science, such a practice is called "hypothesis formation," where the hypothesis is treated as a question to be defined, validated and refuted or confirmed by the data.

Microsoft Surface, new way to play arround

Microsoft Surface is a new technology based on touch screen and very useful for data presentation in Dynamic way. This new feature is very important for Business Intelligence as well, so the entire presentation in the last Conference was dedicated to BI and Data Mining.
Reference link:
http://www.youtube.com/watch?v=V94EVrp9nWk

Business Intelligence

This Blog will contain information about Business Intelligence, Data Mining, Data Modeling and Data Science including tutorials, white papers, important updates and business cases mostly based in Microsoft platform.
Our intention is open a discussion blog where experts can talk generally about Business Intelligence or can exchange views for particular problems that they experienced. We are going to talk about different BI platforms their advantages and disadvantages, against Microsoft Platform.
Analytics will be the main topic, SSAS will be the most discussed tool and SQL/MDX/DMX will be the most used scripts to explain many of the problems that BI Professionals face every day.
MDX and DMX will be part of this blog too. Advanced calculations that we can handle with MDX and problems for improving time in reporting large data warehouse calculations over dimensions.
Dimensional databases vs relational, OLAP Cubes, algorithms for time improvement will rich our Blog.
You will be updated with podcast, white papers, analysis and links that are important to our auditorium.

Best regards,
Besim Ismaili
Creator of the Blog