Data Visualization

"Basically... porn is everywhere," says a new report from the Children's Commissioner. We try to find out just how much – and what the information we do have says about the UK's habits• Read analysis of the report from Alexandra Topping• More data journalism and data visualisations from the GuardianUnsurprisingly, on the Datablog we often write articles about data when we have data. But some topics, like pornography, aren't conducive to statistical analysis, no matter how important many claim they are. Despite these challenges, a report released today has sought to assess children and young people's exposure to pornography and understand its impact. Led by Middlesex University and commissioned by the Children's Commissioner, this was a rapid evidence assessment - completed in the space of just three months as part of a much larger ongoing inquiry into child sexual exploitation.The report found that a "significant proportion of children and young people are exposed to or access pornography", and that this is linked to "unrealistic attitudes about sex" as well as "less progressive gender role attitudes (e.g. male dominance and female submission)".Though the report makes these and other important conclusions, you'll notice that numbers are conspicuously absent in its language. One reason is that its findings were not based on primary research but a literature review that began with 41,000 identified sources and concluded by using 276 of those that were deemed relevant. Several of the articles in the references are however quite dated - some published as far back as 1980. That doesn't necessarily mean that the findings are irrelevant. Rather, it shows just how difficult it is to analyse an entertainment activity/hobby/addiction (delete as appropriate) that is as prevalent as it is provocative - or at least we think. In fact, for a phenomenon that is believed to be so widespread and so regularly features in debates about the state of British society, there is virtually no accurate data on pornography. In February of this year, I contacted academics that conduct studies on pornography (though many of these look at its impact, rather than its scale) as well as 'industry experts'. One of those was someone working at Erotic Trade Only, which describes itself as "UK's leading adult industry magazine". I didn't get very far. The industry expert replied "there are no figures, unfortunately, and with the industry continually changing its delivery vehicles and routes to market that is unlikely to change anytime soon". The academic meanwhile pointed out, "much of what's out there is either self-reported by the industry or by its opponents, neither of which is particularly reliable". That's not even to mention that while a precise definition of 'pornography' remains elusive, so too do efforts to define where the boundaries of its industry stop and where they begin.So what can be said? Well, like shopping, dating and programme viewing, the proportion of pornography online is growing compared to that which is offline. Maybe then, it's useful to look at some trends on Google. Trends like, for instance, the frequency of porn as a search term which, as the interactive graph below demonstrates, is becoming more common. This, as the next interactive shows, is a trend which is quite equally spread across the country. England tops the list for online searches, Scotland has 97% the amount England does, Wales 87% and residents of Northern Ireland produce 3/4 the amount of web searches for the term 'porn' that England does. But since Google Trends doesn't show absolute values, we're still no closer to understanding the scale of porn or its place in British cultural life relative to other trends. Well, what about if we were to use data on the most Googled terms in the UK in 2012 (Euro 2012 tops the list) and compare it to searches for 'porn'? It appears that even at the height of football madness in June, porn was by far a more interesting topic for those

How likely are you to see a piece of art in a London gallery created by a woman? An art audit by the East London Fawcett Group has attempted to find out• Get the data• More data journalism and data visualisations from the GuardianAn audit of more than 100 commercial galleries in London has found that only 5% represent an equal number of male and female artists. East London Fawcett's (ELF) art audit also found that not a single woman appeared on the top 100 auction performances list in 2012. The audit which looked at works from April 2012 – April 2013 gathered data on 134 commercial galleries in London, which collectively represent 3,163 artists. Of this total, just 31% of the represented artists were women, with 78% of the galleries representing more men than women. Kira Cochrane writes today:Here's a teaser. How many women artists featured in the top 100 auction sales, ranked by price, last year? Gemma Rolls-Bentley, an independent curator, decided to find out. One day, not long ago, she sat down with the 2012 list, "and spent a couple of hours writing M next to the artists. I got to the end and there wasn't a single F." Some of those artists were alive, some were dead, all were highly valued - clearly considered "great" or "genius" - and all were men.ELF, the East London branch of the Fawcett Society, also turned their attention to gender representation in solo shows featured in the exhibition programmes of 29 non-commercial galleries in London. Nearly a third of the galleries presented no female solo shows during this period and only one of the gallery programmes featured all female shows.Public art also came under scrutiny from the ELF audit. Of the 386 public works of art that were recorded in Westminster and the City of London, a mere 8% were created by female artists. That may not come as much of of a surprise if you take into account that a large proportion of the pieces of art date back many years.So if we look at the modern day, do female artists fare better? A quarter of the artists selected for the Fourth Plinth, situated in the northwest corner of Trafalgar Square, were female, as Kira Cochrane writes today "far from brilliant, but much better than those other statistics for public art". Frieze Art Fair, an annual showcase of leading international artists, provided some interesting results. 27.5% of the artists represented at the art fair in 2012 were women. The results are a reflection of a survey that took in 3,441 artists across 135 international galleries that were represented in the commercial section of Frieze Art Fair 2012. However, 23.3% of solo exhibitions hosted by commercial galleries in the capital during Frieze week presented female artists – an increase on the 11.6% figure that Laura McLean- Ferris found in 2008 when she conducted a similar survey. Gemma Rolls-Bentley, arts director at ELF, is optimistic despite the gender gap displayed by the results: The ELF art audit results provide statistical evidence that gender inequality still persists in London's art world. However, these results also demonstrate that significant positive progress has and is being made. By raising awareness of the challenges specific to female artists, we hope that the campaign will widen the dialogue around this issue and that as a result the gender balance will continue to improve. The art audit's message is one of optimism.Campaign group, UK Feminista, published results in 2010 of a similar piece of research looking at gender inequalty in the art world. They found that 83% of the artists in the Tate Modern and 70% of the artists in the Saatchi Gallery were male.But the gender gap is the reverse when you look at university stats. In her Datablog piece examining the gender gap at universities by institution and subject, Rebecca Ratcliffe found that 61.7% of the undergraduate creative arts and design intake in UK universities in 2011-12 was female. So why are so few female artists being represented in art galleries? We'd lo

Since 2011, net migration has been on the decline due to falling numbers of immigrants. What are the other key trends behind these often controversial statistics? • Get the data• More from the Datablog on immigration• More data journalism and data visualisations from the GuardianIt's that time of year again: the release of migration statistics. Many will be keen to inspect how close these numbers come to the Conservative's target to reduce net migration to 100,000 by 2015 when they will again face the vote. The latest numbers from the Office for National Statistics show that net migration was 153,000 in the year ending September 2012, compared to 242,000 the previous year.Alan Travis has more on the story here, including this comment from the immigration minister, Mark Harper:The figures show we have cut out abuse while encouraging the brightest and best migrants who contribute to economic growth, with a 5% increase in the number of sponsored student visa applications for our world-class universities, and a 5% increase in the number of visas issued to skilled workers.Latest totalsThough the data for the twelve months to September 2012 is still provisional, it suggests that half a million people immigrated to the UK. This represented a 14% reduction from 581,000 immigrants (or 'inflow' as it's named in the data).Long-term emigration meanwhile is rising - up 2% from 339,000 in the year ending September 2011. Though immigration and emigration have moved in opposite directions over the past year, the changes have not been enough to offset one another meaning that net migration remains a positive value. Reasons for comingA critical piece of information for policymakers - wherever they sit on the political spectrum - is the reason given by those who have decided to enter or leave the UK. Here, trends are just as visible as elsewhere. Formal study has been the most common reason given by those immigrating, followed by a work-related motivation. Almost half (190,000) of long-term migrants state study as their reason for coming, though these individuals often receive less attention than the 62,000 who come to the UK to accompany or join a family member already here.Also, often overlooked is the fact that the majority of those who state work as their reason for coming (175,000) are also able to state that they have a definite job. A smaller fraction, 38%, come to the UK in search of employment. 58% of those leaving the UK cite work as a reason for doing so - of these, 64% have a definite job waiting for them in their destination of choice, the remainder state that they are heading off in search of work. Changes in motivation appear to coincide with the financial crisis - more people leaving the UK cited work as a reason for their decision after 2007. Similarly, 2007 was the first year in which more immigrants cited study rather than work as a reason for coming - a trend which has continued ever since. Citizenship trendsFinally, who exactly is arriving and (probably a less controversial question) where are people leaving the UK going to? Well, the Office for National Statistics summarise these numbers using the following headings:• British• EU• EU 15 (EU countries as constituted between 1 January 1995 and 1 May 2004)• EU 8 (eight Central and Eastern European countries that acceded to the EU on 1 May 2004)• All non-EU - which is comprised of• Old commonwealth• New commonwealth• Other foreignWhen the numbers are broken down by citizenship, some of the most striking trends are to be seen among non-British citizens. For example, net-migration of non-British citizens has fallen by 25% from 303,000 in 2011 to 228,000 in 2012. Net-migration of EU citizens remained more stable, falling by 12% to 66,000 in 2012 compared to the previous year. Non-EU destinations remain slightly more appealing to those leaving the UK - 78,000 headed to EU countries compared to 104,000 leaving to places outside the European Union.Below are biannual and, where avail

A decade of global data attempts to analyse the details of bullying. But what can the figures really tell us about an issue that is so difficult to record?• Get the data• More data journalism and data visualisations from the GuardianThe evolution of social media and mobile communication may have made it easier than ever for young people to share but they also create an environment that can make bullying "inescapable and even more threatening than ever before" according to a new report by Child Helpline International (CHI). CHI, a network of government and civil society organisations, operates 173 child helplines in over 142 countries and in the past 10 years has collated a database of more than 126m contacts by children and adults on behalf of young people from its member helplines. The 126m refers to the number of conversations that have taken place between a child or young person and a counsellor of a child helpline somewhere in the world, on any subject a child or young person wanted to talk about. The database has collated data through any form of communication used by child helplines including telephone, chat, SMS, message boards, walk-in centres and outreach activities. Of the 126m, nearly 4m have been about abuse and violence, including categories such as bullying, emotional abuse, physical abuse, sexual abuse and neglect. And since the CHI started collecting data on cyberbullying in 2011, more than 27,000 contacts have been recorded on this subject.On average, every child helpline in the world receives nine contacts from children and young people per day who are suffering the effects of bullying, according to CHI.Of course the results of this report don't tell the whole story, for every child that seeks advice by contacting a helpline, there are many more that either do not have the access, confidence or privacy to do the same. As a result, gathering data on the number of children suffering from bullying has never been simple, but the CHI's analysis helps give an insight into a global problem affecting many.NSPCC statistics on bullying collated from government reports and research suggest that almost half of children and young people have been bullied at school at some point in their lives. The NSPCC also report that 38% of young people have been affected by cyberbullying.Figures from a 2011 report by the Department of Education (DfE) also show that girls are twice as likely to experience persistent cyberbullying than boys. This trend was also apparent in CHI's analysis - the number of girls contacting them about cyberbullying was slightly higher than boys, although 90% of those contacting child helplines "hesitated to disclose their gender to protect their identity and maintain their anonymity after having suffered online abuse".The overall proportion that were not willing disclose their gender for recording purposes was 71%. Ofcom research, published in 2008, showed that almost half of children aged 8-17 who use the internet had set up their own profile on a social networking site. The Ofcom research also reported the following observation: It also appears likely that when children receive hostile, bullying or hateful messages, they are generally ill-equipped to respond appropriately or to cope with the emotional upset this causesSo what else does the release by CHI show? Well, the number of contacts received in 2012 was more than double those received in 2006, but as CHI note, the rise in contacts could also be associated with growing awareness of bullying amongst children. Bullying can take many forms but analysis on information gathered since 2011 has highlighted four major categories; emotional, physical, exposure and theft. Almost half of the contacts on bullying could be be categorised as emotional bullying and nearly a quarter as physical abuse. Instances where young people have been either exposed to bullying as a witness or have had belongings stolen accounted for 12.5% of contacts each. Emotional bullying

How bad is the deficit really? We bring you all the data going back to the 1940s• Get the dataHow bad is Britain's deficit? The latest set of figures show that Britain's deficit was £2.5bn lower in April than the same month a year earlier.The Office for National Statistics said public sector net borrowing came in at £85.1 for the 2012-13 financial year. That's a £35.8bn improvement on the £120.9bn in the previous year..Heather Stewart writes today: George Osborne received a boost on Wednesday with news that the deficit was £2.5bn lower in April than the same month a year earlier, boosting hopes that his plan to repair the UK's public finances is back on track.We have the complete set of data on Government borrowing, all the way back to the 1940s. All political parties have faced their fair share of debt through the years - almost as if the economic climate has its own life independent of who is managing it. UK public debtWhat is the deficit? When the ONS talks about the deficit, they take a simple measure - the gap between what's coming into the government in taxes and receipts versus what's being spent. Most commentators look at net borrowing as the deficit figure, because it includes investment spending. It's different to the national debt - which is the total the country owes.So last month the budget was in deficit. Here are the key facts for November - if you exclude the temporary effects of the financial interventions in the banks:• Public sector current budget deficit was £5.6bn in April 2013; this is a £2.5bn lower deficit than in April 2012, when there was a deficit of £8.2bn. • Public sector net borrowing (PSNB ex) was £6.3bn in April 2013; this is £25.4bn higher net borrowing than in April 2012, when net borrowing was £-19.1bn. • For 2012/13, public sector net borrowing (PSNB ex) was £85.1bn; this is £35.8bn lower net borrowing than in 2011/12, when net borrowing was £120.9bn. • For 2012/13, central government net cash requirement was £109.7bn; this is £16.8bn lower net cash requirement than in 2011/12, when net borrowing was £126.5bn. • In 2012/13, public sector net borrowing and public sector current budget deficit are reduced by £6.4bn as a result of cash transfers from the Bank of England Asset Purchase Facility Fund to Government. • In 2012/13, public sector net borrowing and public sector net investment are reduced by £28.0bn as a result of the transfer of the Royal Mail Pension Plan in April 2012. • After removing the effects of the transfer of the Royal Mail Pension Plan and the transfers from the Bank of England Asset Purchase Facility the first 2012/13 estimate of public sector net borrowing is similar in level to last year's borrowing at £119.5bn, £1.4bn lower net borrowing than in 2011/12. • Public sector net debt was £1,185.3bn at the end of March 2013, equivalent to 75.2% of gross domestic product (GDP).The ONS data below shows monthly, quarterly and annual debt and deficit - what can you do with it?Download the data• DATA: download the full list as a spreadsheetMore dataMore data journalism and data visualisations from the GuardianWorld government data• Search the world's government data with our gatewayDevelopment and aid data• Search the world's global development data with our gatewayCan you do something with this data?• Flickr Please post your visualisations and mash-ups on our Flickr group• Contact us at data@guardian.co.uk• Get the A-Z of data• More at the Datastore directory• Follow us on Twitter• Like us on FacebookGeneral election 2010Economic policyEconomic growth (GDP)Tax and spendingPublic financeOffice for National StatisticsGovernment borrowingLiberal-Conservative coalitionAlistair DarlingSpending review 2010Office for Budget ResponsibilityJulia KolleweSimon Rogersguardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

The Rendition Project has spent three years creating an interactive guide to CIA rendition flights of terrorist suspects, containing more than 11,000 rows of dataJames Ball

The Rendition Project, a collaboration between UK academics and the NGO Reprieve, has produced one of the most detailed and illuminating research projects shedding light on the CIA's extraordinary rendition project to date. Here's how to use it.• See The Rendition Project interactive hereThe Rendition Project, run by UK academics, has collaborated with the NGO Reprieve to produce one of the most detailedand illuminating research projects shedding light on the CIA's extraordinary rendition project to date.In a single interactive graphic, it shows in great detail the data behind every confirmed and suspected rendition flight, and then – as it's also intended as a tool to fuel further research and digging – a huge number of other flights of the planes linked to rendition. In total, the data powering the graphic runs to more than 11,000 lines.Of course, that means that the graphic's complex, and so we've provided a guide on how to read and interpret it below. A key caveat is that not every flight contained within the interactive is tied to rendition: some are suspected rendition flights, others are simply flights from planes with tail numbers that were used on suspected rendition flights.It's also important to note that just because a particular company owned or operating a plane believed to have been involved in rendition, it does not necessarily follow that the company itself was involved or even aware of those activities. In some cases, it's unclear whether the airline companies would have been aware of the purpose of the flights.A wealth of supporting data and research – including original documents – has been published directly on The Rendition Project's website.Now, here's how to get the most from the interactive:Picking what to look atBy default, the graphic shows a huge tangle of different flight routes – it's displaying information on the 1,500 or so flights marked as significant within the data: the ones with some suspected involvement in rendition (those doing advanced research can toggle this off using the "key circuits only" drop-down menu).The graphic's easiest to use if this is narrowed down. The graphic is broken down into "circuits" of flights: a full trip made up of several different legs. The screenshot used to illustrate this post represents a round-the-world circuit made up of a number of different airport-to-airport trips.Circuits can include original journeys from America, R&R stops in the Caribbean, refuelling stops, and the central rendition journeys themselves.The menu on the left-hand side of the graphic gives a range of ways the information can be narrowed down: trips which only take in certain airports can be picked, or particular companies, or particular individuals known to have been targets of rendition. The date range can also be selected using the sliding toggles.Hitting the large "SEARCH" button at the bottom-left will then update the map with the new settings.What the different colours meanDifferent individual flights are colour-coded by their significance.The simplest flights are marked in grey. These are legs of the flights where the researchers had no reason to believe there was any detainee aboard the aircraft. These mark refuelling stops, planes getting into position, R&R stops, or similar.At the other end of the scale, strong red lines mark a flight designated a "rendition flight". These are flights where the researchers are as near as possible as investigators on these topics can be to certain that a detainee – often a named detainee – was aboard the plane. These are backed by a wealth of evidence.Paler red lines mark "highly suspicious" or "suspicious" flights – one where there is evidence – often strong – to believe a detainee was aboard a given flight, but where the researchers are not quite so confident. Some of the "suspicious" flights have been flagged because of very similar routes or timings to flights tied to rendition, rather than specific evidence on that particular flight.The

About 35,000 meteorites have been recorded since 2500 BC, and a little over 1,000 of them were seen while they fell, based on data from the Nomenclature Committee of the Meteoritical Society. Carlo Zapponi, a data visualization designer, visualized the latter in Bolides. We saw a mapped version of this data a while back, but Bolides takes a time-based approach. A bar chart shows the number and volume of meteorites that have been seen over time, and on the initial load, you get to watch the meteorites fall, one bright orange fireball at a time.

It's not just UK homeowners that are considering downsizing, one of the country's biggest landowners is doing it too: the government. We look at how the state estate has changed since 2011Get the dataMore data journalism and data visualisations from the GuardianThe government has been criticised in the past for its inefficient use of the buildings it owns. From the plans for small businesses to take over the disused premises to suggestions that they be turned over for community use, there has been pressure from a number of quarters that the government could save space and money.It appears that they've responded to that pressure. Data released by the Cabinet Office today shows that the government's estate has shrunk 10% from 10.3m square metres in 2011 to 9.2m square metres in January of this year. The data unfortunately does not include information on who these properties were sold to (or, if they weren't owned, who they are no longer being rented or leased from) what they are now being used for or the amount of money spent/saved per building. It does however show that, taken together, 190 government buildings split between 31 different departments or organisations shrunk by 1,081,535 square metres between 2011 and 2013.That does not mean that every government building shrunk. In fact, 39 of those included in the data actually increased in size and 31 didn't change at all. The biggest changes were:HM Revenue and Customs shaved off a whopping 205,000 square metres from its estate - cutting it down to a mere 1.1 million square metresThe next biggest shrinkage was for the Ministry of Justice's National Offender Management Service (NOMS) which lost 117,000 square metresHer Majesty's Court Service by contrast witnessed the largest growth (42,000 square metres) closely followed by the British Transport Police Authority who acquired new premises 41,000 square metres in sizeThe State of the Estate report also looked at other features of the government estate. It found that 764 buildings were constructed after 1990 out of the 2,719 government building that had their age profiled. 182 of those buildings date back to pre-1900.The report also looks at how sustainable the various government buildings are and finds that most are performing well in reducing greenhouse gas emissions, water and waste. And just like responsible homebuyers, the report looks at the energy performance of new government acquisitions. Topping the list are the new Ministry of Justice buildings in Caernarfon and Llandrindod Wells.The largest vacant public buildings were to be found in the North West and the South West where 41,00 and 40,000 square metres respectively were not in use.Though there isn't a breakdown per building, the report does show which government departments and organisations have the largest annual running costs. At £604m per year, it's perhaps unsurprising that the department dealing with prisons tops the bill - the Ministry of Justice. Speaking about the changes, Francis Maude, Minister for the Cabinet Office said:We are pushing forward our ambitious programme to sell property we no longer use and get out of expensive rentals.Since the last general election we have saved the taxpayer over a £1 billion and made space available in desirable central locations for redevelopment. This is not only about changing the way we work but generating an economic boost. The changes over the past two years are part of the Government Property Unit strategy, the next stage of which concentrates on putting the existing space to better use. What do you think about these changes? Do they reflect a sensible rationalisation of otherwise inefficient resources? Or is downsizing an inevitable consequence of public sector job cuts? Share your views via Twitter @MonaChalabi or @GuardianData or by posting a comment below.Download the data• DATA: download the full spreadsheetCan you do something with this data?• Flickr Please post your visualisations and mash-ups on our Flick

Reader Jack S. sent over this chart (link): The first problem readers encounter with this image is "What is MMI?"  I like to think of any presentation as a set of tearout pages. Even if the image is part of a book, or part of a deck of slides, once it is published, the writer should expect readers from tearing a sheet out and passing it along. In fact, you'd love to have people pass along your work. This means that when creating a plot such as this, the designer must explain what MMI is in the footnote. Yes, on every chart even if every chart in the report deals with MMI. MMI, I'm told, is some kind of metric of health care cost. *** What a mess. They are trying to use the metaphor of "measuring one's temperature", which I suppose is cute because MMI measures health care costs. Next, the designer chose to plot the index against the national average as opposed to the dollar amount of MMI. This presents a challenge since the thermometer does not have a natural baseline number. This is especially true on the Fahrenheit scale used in the U.S. Then, a map is introduced to place the major cities. The bulb of each thermometer now doubles as a dot on the map. This step is mind-boggling because the city labels aren't even on the map. So if you know where these cities are, you don't need the map for guidance but if you don't know the locations, you're as hopeless as before. How the data now gets onto the complex picture requires some deconstruction. First, start with a bar chart of the relative index (the third column of the table shown above). Then, chop off the parts below 85 (colored gray). Next, identify the cities that are below the national average (i.e. index You can see this by focusing only on the chart above the map. In other words, this part: To get from here to the version published, add a guiding line from each bar to the dot on the map for the corresponding city. Notice that a constant-length portion of each bar has been chopped off, and now each bar is augmented by some additional length that varies with the distance of the bar chart from the geographical location of the city as shown on the map below. For instance, Miami, which is furthest south, has the biggest distortion. *** The choice of 85 as a cutoff is arbitrary and inexplicable. If we really want to create a "cutoff" of sorts, we can use 100, which represents the national average. By plotting the gap between the city index and the national index, effectively, the percent difference, we also can use the sign of the difference to indicate above/below the national average, thus saving a color. *** One of the most telling signs of a failed chart is the appearance of the entire data set next to the chart. That's the essence of the self-sufficiency test.