Programming

2013/05/25 -- Edward Capriolo
2013/05/25 -- Edward Capriolo
30 minutes ago
The highlight of this system is a clear separation of the cube computation engine and the query serving engine of this OLAP system. Another newness is the use of a key value store, Voldemort, to fetch the results of queries. Contrary to ...
The highlight of this system is a clear separation of the cube computation engine and the query serving engine of this OLAP system. Another newness is the use of a key value store, Voldemort, to fetch the results of queries. Contrary to conventional designs where the cube computation engine and the query serving infra are coupled tightly, Avatara chooses to separate these two aspects. The offline batch engine is based on Hadoop. A set of MapReduce jobs transform the data into cubes. These cubes are then bulk loaded into the key value store for fast online access. The batch processing pipeline consists of three phases: preprocessing, projections and joins and cubification. Each of these phases is a collection of MapReduce jobs and carried out executed sequentially. After data is preprocessed in the first phase a star schema is modeled in the second phase. The engine projects and joins the fields from the fact and dimension tables and writes the output to a temp location. The third phase is when the cubification happens. This phase generates a number of small cubes as MOLAP blobs (multidimensional array) containing dimensions and measures. The storage format is optimized such that the query engine can retrieve this data in a single disk fetch. When clients issue queries it is intercepted by the online query engine component. The engine reads data from the key value store. The query engine models a SQL like syntax and supports operations such as select, where and group by along with some math operations. Since the system is built to keep query response times very low it disallows joins from being performed online. The discussion section (3.4) of the paper best summarizes the design trade offs. Worth inlining verbatim – Avatara provides ?exibility in terms of where cube materialization can happen: operations such as sum, average, order, or limit can be performed of?ine or online. With more of?ine aggregation, online queries will be faster, but naturally less ?exible. For example, after a developer speci?es the granularity of a time dimension to be at the “week” level in the of?ine phase, future online aggregations can only happen at equal or coarser levels such as “weeks” or “months”. For WVMP, partial materialization happens of?ine to roll up pro?le visits into a weekly aggregation level; the remaining materialization happens online because the cubes are small. This also enables us to introduce new query patterns by selecting a different set of dimensions without recomputing any cubes. Link to the paper.
about 1 hour ago
This past week 6 new mashups were added to our mashup directory and 6 different APIs were used to build them. Some of the newer or less frequently seen APIs include PriceSpin and Primal. The most often used APIs this week are foursquare...
This past week 6 new mashups were added to our mashup directory and 6 different APIs were used to build them. Some of the newer or less frequently seen APIs include PriceSpin and Primal. The most often used APIs this week are foursquare, Google Maps and PriceSpin. And the most commonly used types of APIs were Mapping (1 APIs, 2 mashups), Music (1 APIs, 1 mashups) and Shopping (1 APIs, 1 mashups). The list below shows which APIs were used by which mashups: Echo Nest used in Bluth Radio foursquare used in HangoutLater Google Maps used in Companies Near Me, GeoGuessr Instagram used in Worldcam PriceSpin used in ShopSmart Brazil Primal used in ShopSmart Brazil Mashups of the day:And each day there is one mashup selected to be Mashup of the Day. Here are last week’s winners: Bluth Radio Companies Near Me GeoGuessr HangoutLater ShopSmart Brazil Worldcam Sponsored by
about 2 hours ago
While fine art is a subjective field, graphic design is more formulaic in its fundamentals. An effective design should make people feel a certain way and take a certain action. In this article I’d like to share Gravity Switch’s web desig...
While fine art is a subjective field, graphic design is more formulaic in its fundamentals. An effective design should make people feel a certain way and take a certain action. In this article I’d like to share Gravity Switch’s web design principles and our thinking behind them.
about 3 hours ago
At pennies per virtual machine-hour, the economics of cloud computing are both compelling and daunting to replicate. Whether you are building your own cloud infrastructure, building a public cloud or choosing a cloud service, there are k...
At pennies per virtual machine-hour, the economics of cloud computing are both compelling and daunting to replicate. Whether you are building your own cloud infrastructure, building a public cloud or choosing a cloud service, there are key strategy and technology decisions that make the difference between success and failure. In his General Session at the 12th International Cloud Expo, Jason Waxman, VP in the Intel Architecture Group and general manager of the Cloud Platforms Group within Intel's Datacenter and Connected Systems Group, will share industry best practices for deploying cloud infrastructure that maximize the benefits of cloud economics, agility and interoperability. Learn how Intel is working with industry leaders to deliver open, secure and efficient cloud computing based on optimized compute, networking, storage and software technology and what are key tools and resources to help you achieve your cloud computing goals.read more
about 3 hours ago
You're getting pitched every day from your legacy enterprise software and hardware vendors about "cloud." They're doing an amazing job of convincing your CIO and CTO about what cloud is and how you should use it. The reality is...
You're getting pitched every day from your legacy enterprise software and hardware vendors about "cloud." They're doing an amazing job of convincing your CIO and CTO about what cloud is and how you should use it. The reality is they're defending their shrinking market share and keeping you on the legacy treadmill for as long as they can by selling you solutions that aren't "cloud." In her session at the 12th International Cloud Expo, Niki Acosta, Cloud Evangelista for Rackspace, will talk through the five dirty little secrets your legacy IT vendors won't tell you about cloud. Attendees will leave with an understanding of how you should really be thinking about building enterprise apps for the cloud while embracing the fundamental shift away from legacy IT.read more
about 3 hours ago
The technology infrastructure of today’s business landscape has undergone dramatic shifts in recent years. The consumerization of IT, the Bring Your Own Device (BYOD) movement, and of course the cloud have revolutionized the way corporat...
The technology infrastructure of today’s business landscape has undergone dramatic shifts in recent years. The consumerization of IT, the Bring Your Own Device (BYOD) movement, and of course the cloud have revolutionized the way corporations manage data, complete transactions and communicate with employees and customers. Organizations of all sizes are moving their document and email management needs to the cloud, with increasing numbers opting for a private cloud solution, a proprietary computing architecture that provides hosted services behind an organization's firewall. read more
about 3 hours ago
PHP
Recent releases from the Packagist:mandrill/mandrill (1.0.30) API client library for the Mandrill email as a service platform sabre/vobject (3.0.0-alpha3) The VObject library for PHP allows you to easily parse and manipulate iCalendar an...
Recent releases from the Packagist:mandrill/mandrill (1.0.30) API client library for the Mandrill email as a service platform sabre/vobject (3.0.0-alpha3) The VObject library for PHP allows you to easily parse and manipulate iCalendar and vCard objects pinano/datatables-bundle (1.9.4) Symfony2 bundle for wrapping the famous dataTables jquery plugin by SpryMedia pinano/select2-bundle (3.4.0) Symfony2 bundle for wrapping the famous select2 jquery plugin by @ivaynberg phundament/p3widgets (0.11.8) Widget Manager simplon/db (1.2.3) Simplon DB Libraries kr4y/profiler (1.0.2) Profiler for Laravel 4 zweifisch/zf (v0.2.9) a micro php web/cli framework/router phundament/p3pages (0.11.8) Page Manager patchwork/utf8 (v1.1.8) UTF-8 strings handling for PHP 5.3: portable, performant and extended piwik/piwik (1.12-b19) Open Source Real Time Web Analytics Platform gocardless/gocardless (0.4.1) The PHP client library for the GoCardless API hostnet/form-twig-bridge (0.5) Allows usage of the Symfony 2 form framework, rendered in twig, without using all of Symfony 2! simplon/helper (0.6.0, 0.5.11) General helpers which come-in helpful. gabriel403/g403-session-db (1.0.2) Extension to ZF2 to allow sessions to be stored in the database. willdurand/expose-translation-bundle (0.3.0) A pretty nice way to expose your translation messages to your JavaScript. vojtech-dobes/nette-forms-gpspicker (v1.0.7) Google Maps based picker of coordinates for Nette Framework. mjohnson/uploader (4.0.15) File uploader and validation plugin for CakePHP.
about 4 hours ago
Economic uncertainty surrounding the U.S. government sequester, European debt crisis and weakening GDP in China has resulted in volatile spending patterns across most segments of the market. According to the latest market study by Intern...
Economic uncertainty surrounding the U.S. government sequester, European debt crisis and weakening GDP in China has resulted in volatile spending patterns across most segments of the market. According to the latest market study by International Data Corporation (IDC), as a result of the current economic climate, business technology spending was slightly below expectations in the second half of 2012 and first quarter of 2013. IDC now projects worldwide IT spending growth of 4.9 percent this year in constant currency, down from the previous forecast of 5.5 percent growth -- and representing a slowdown from the 5.6 percent growth recorded in 2012. As a result, IT spending will reach $2.06 trillion in 2013. Including telecom services, ICT spending will increase by 4.5 percent to $3.7 trillion. So, what are the key trends that are shaping the updated global forecasts? Deteriorating PC Shipments The reduction in IDC’s overall forecast for 2013 is largely driven by rapidly deteriorating PC shipments since the second half of 2012. IDC now expects PC spending to decline by 3 percent in constant currency this year, representing a third successive year of declining PC revenues. The shift to mobile devices remains a key driver for overall tech spending growth. Excluding mobile phones and tablets, worldwide IT spending increased by only 2.8 percent in 2012 and is forecast to grow by just 2.6 percent this year. Worldwide spending on smartphones will increase by 17 percent in 2013 while tablet spending will grow by 32 percent. The combined growth rate for PCs and tablets, meanwhile, will remain stable in the range of 4-5 percent. Cloud Services Cannibalizes Software and IT Services Just as tablets are cannibalizing PC spending, so the growth of managed cloud services continues to cannibalize commercial software and IT services. Software spending in the U.S. grew slightly slower than forecast in 2012, and IDC has consequently reduced the U.S. software forecast to 6 percent growth for 2013 (from 7 percent). IT services demand remains stable, but the pass-through from capital spending and software deployment remains tepid by historical standards. IDC now forecasts growth of 5.6 percent in worldwide software spending in 2013 (constant currency), and 3.8 percent in IT services. Decline in Server Revenues IDC's assessment also suggests a decline in overall server revenues while storage infrastructure spending will cool somewhat after the major spending cycle of 2011/2012. IDC now projects 2.4 percent growth in worldwide storage hardware revenues this year, down from 6.1 percent growth in 2012. Network infrastructure investment was strong in 2012, as many carriers invested in the deployment of LTE networks, but this will also cool in 2013. Service provider spending on network equipment will increase by 1.1 percent this year, compared to 5.8 percent in 2012. Enterprise network spending should remain more stable, projected to post growth of 6.8 percent. The Global Networked Economy Emerging markets are still the engines of growth for Worldwide IT spending, with strong trends continuing in markets such as India and Brazil in recent months. The weakest-performing geographies will be Western Europe and Japan, where slow economic growth is inhibiting IT spending while the U.S. market remains fragile in the context of political uncertainty. "It's all about the economy," said Stephen Minton, Vice President at IDC. "Our surveys confirm that underlying demand for IT products and services remains strong, but that businesses are once again being forced to delay new projects or investments in the face of longer decision-making cycles and a lack of short-term visibility. This storm could pass quickly, if governments in the U.S., Europe, China and Japan succeed in steering their ships towards calm waters in the second half of the year." That being said, it's the savvy forward-looking executive leaders who continue to invest
about 5 hours ago
Alex Papadimoulis shares his thoughts on distribution vs delivery, decoupling infrastructure (pull) from application (push) deployments and keeping delivery systems simple, especially for web scale applications. In particular Alex descri...
Alex Papadimoulis shares his thoughts on distribution vs delivery, decoupling infrastructure (pull) from application (push) deployments and keeping delivery systems simple, especially for web scale applications. In particular Alex describes three different types of roll-outs: Live, Rolling and Parallel and their applicability (cloud-based delivery vs in-house servers). By Alex Papadimoulis
about 5 hours ago