This is a short webscraping project where I obtain the top 1000 tracks of all time on A State of Trance radioshow hosted by Armin van Burren.
# A State of Trance Top 1000 Of All Time
# List Released on January 11, 2021
# Load libraries:
library(rvest)
library(dplyr)
library(tidyr)
library(stringr)
This code chunk below is reading the html page and extracting the tracks into the songs variable.
# Link:
url <- "https://www.astateoftrance.com/news/armin-van-buuren-reveals-all-time-a-state-of-trance-top-1000-list/"
# ASOT Top 1000:
url_page <- read_html(url)
# Obtain div tag with class columns:
songs <- url_page %>%
html_nodes("[class='columns']")
Tracks are found in the p tags of the songs.
# Tracks are found in the p tags of songs. More specifically in index 6 and 7.
tracks <- songs %>% html_elements('p')
tracks <- tracks[c(6, 7)] %>% html_text2()
# Obtain top 50, extract from nested list
top_50 <- str_split(tracks[1], pattern = "\n")
top_50 <- top_50[[1]]
top_50## [1] "1 Armin van Buuren feat. Susana – Shivers "
## [2] "2 Tiësto – Adagio For Strings "
## [3] "3 GAIA – Tuvan "
## [4] "4 Delerium feat. Sarah McLachlan – Silence (Tiësto’s In Search Of Sunrise Remix)"
## [5] "5 Armin van Buuren – Communication (Part 3)"
## [6] "6 Gouryella – Anahera "
## [7] "7 RAM – RAMsterdam (Jorn van Deynhoven Remix)"
## [8] "8 Dash Berlin with Cerf & Mitiska & Jaren – Man On The Run "
## [9] "9 Gareth Emery feat. Christina Novelli – Concrete Angel "
## [10] "10 RAM & Susana – RAMelia (Tribute To Amelia) "
## [11] "11 Above & Beyond feat. Richard Bedford – Sun & Moon (Above & Beyond Club Mix)"
## [12] "12 Gareth Emery & STANDERWICK feat. HALIENE – Saving Light "
## [13] "13 4 Strings – Take Me Away (Into The Night) "
## [14] "14 Armin van Buuren feat. Justine Suissa – Burned With Desire (Rising Star Vocal Mix)"
## [15] "15 Rank 1 – Airwave "
## [16] "16 Paul van Dyk – For An Angel (PvD E-Werk Club Mix )"
## [17] "17 Above & Beyond pres. OceanLab – Satellite "
## [18] "18 Motorcycle – As The Rush Comes (Gabriel & Dresden Sweeping Strings Mix"
## [19] "19 Aly & Fila feat. Jwaydan – We Control The Sunlight "
## [20] "20 Armin van Buuren feat. Sharon Den Adel – In And Out Of Love "
## [21] "21 Veracocha – Carte Blanche "
## [22] "22 Above & Beyond & Gareth Emery pres. OceanLab – On A Good Day (Metropolis) "
## [23] "23 MaRLo & Feenixpawl – Lighter Than Air "
## [24] "24 Dash Berlin – Till The Sky Falls Down "
## [25] "25 Chicane feat. Maire Brennan Of Clannad – Saltwater "
## [26] "26 Above & Beyond with Andy Moor – Air For Life "
## [27] "27 Paul van Dyk feat. Hemstock & Jennings – Nothing But You (PvD Club Mix)"
## [28] "28 Dash Berlin feat. Emma Hewitt – Waiting "
## [29] "29 Roger Shah presents Sunlounger feat. Zara – Lost (Club Mix)"
## [30] "30 GAIA – Empire Of Hearts "
## [31] "31 Andain – Beautiful Things (Gabriel & Dresden Unplugged Mix)"
## [32] "32 Armin van Buuren feat. Trevor Guthrie – This Is What It Feels Like (W&W Remix)"
## [33] "33 Robert Miles – Children (Dream Version)"
## [34] "34 Alex M.O.R.P.H. feat. Sylvia Tosun – An Angel’s Love "
## [35] "35 Gareth Emery feat. Bo Bruce – U (Bryan Kearney Remix)"
## [36] "36 Armin van Buuren feat. Miri Ben-Ari – Intense "
## [37] "37 Key4050 & Plumb – I Love You "
## [38] "38 Richard Durand & Christina Novelli – The Air I Breathe (Club Mix)"
## [39] "39 John O’Callaghan feat. Audrey Gallagher – Big Sky (Agnelli & Nelson Remix)"
## [40] "40 Armin van Buuren feat. Jan Vayne – Serenity "
## [41] "41 Push – Universal Nation "
## [42] "42 Giuseppe Ottaviani – Through Your Eyes "
## [43] "43 Luminary – Amsterdam (Smith & Pledger Remix)"
## [44] "44 Energy 52 – Café Del Mar ’98 (Original Three ‘N One Mix)"
## [45] "45 Aly & Fila – It’s All About The Melody "
## [46] "46 System F – Out Of The Blue "
## [47] "47 Gareth Emery feat. Lucy Saunders – Sanctuary "
## [48] "48 Above & Beyond feat. Zoë Johnston – Good For Me (Above & Beyond Club Mix)"
## [49] "49 ATB – Ecstasy (Clubb Mix)"
## [50] "50 Tiësto – Suburban Train "
# Ranked 51-1000
other_tracks <- str_split(tracks[2], pattern = "\n")
other_tracks <- other_tracks[[1]]
# Combine all:
combined <- c(top_50, other_tracks)
# Dataframe:
df <- data.frame(Rank = seq(1, 1000), Track = combined)
head(df, 20)## Rank
## 1 1
## 2 2
## 3 3
## 4 4
## 5 5
## 6 6
## 7 7
## 8 8
## 9 9
## 10 10
## 11 11
## 12 12
## 13 13
## 14 14
## 15 15
## 16 16
## 17 17
## 18 18
## 19 19
## 20 20
## Track
## 1 1 Armin van Buuren feat. Susana – Shivers
## 2 2 Tiësto – Adagio For Strings
## 3 3 GAIA – Tuvan
## 4 4 Delerium feat. Sarah McLachlan – Silence (Tiësto’s In Search Of Sunrise Remix)
## 5 5 Armin van Buuren – Communication (Part 3)
## 6 6 Gouryella – Anahera
## 7 7 RAM – RAMsterdam (Jorn van Deynhoven Remix)
## 8 8 Dash Berlin with Cerf & Mitiska & Jaren – Man On The Run
## 9 9 Gareth Emery feat. Christina Novelli – Concrete Angel
## 10 10 RAM & Susana – RAMelia (Tribute To Amelia)
## 11 11 Above & Beyond feat. Richard Bedford – Sun & Moon (Above & Beyond Club Mix)
## 12 12 Gareth Emery & STANDERWICK feat. HALIENE – Saving Light
## 13 13 4 Strings – Take Me Away (Into The Night)
## 14 14 Armin van Buuren feat. Justine Suissa – Burned With Desire (Rising Star Vocal Mix)
## 15 15 Rank 1 – Airwave
## 16 16 Paul van Dyk – For An Angel (PvD E-Werk Club Mix )
## 17 17 Above & Beyond pres. OceanLab – Satellite
## 18 18 Motorcycle – As The Rush Comes (Gabriel & Dresden Sweeping Strings Mix
## 19 19 Aly & Fila feat. Jwaydan – We Control The Sunlight
## 20 20 Armin van Buuren feat. Sharon Den Adel – In And Out Of Love
The format before was just rank and track. This extension part will separate the artist and song from the hyphen. One small detail to note is that the hyphen from the A State of Trance list is a different hyphen from my keyboard. You have to copy and paste this hyphen in the sep = argument in separate().
# Separate by hyphen. That is the hyphen from the site (-) not my keyboard hyphen (-):
asot_top_1000_df <- df %>%
separate(col = Track, into = c("Artist", "Song"), sep = '–')
# Remove number in artist column:
# Reference: https://stackoverflow.com/questions/32767164/use-gsub-remove-all-string-before-first-white-space-in-r
asot_top_1000_df$Artist <- str_replace(asot_top_1000_df$Artist, "^[\\S]+", "") %>% trimws()
# Preview table:
head(asot_top_1000_df)## Rank Artist
## 1 1 Armin van Buuren feat. Susana
## 2 2 Tiësto
## 3 3 GAIA
## 4 4 Delerium feat. Sarah McLachlan
## 5 5 Armin van Buuren
## 6 6 Gouryella
## Song
## 1 Shivers
## 2 Adagio For Strings
## 3 Tuvan
## 4 Silence (Tiësto’s In Search Of Sunrise Remix)
## 5 Communication (Part 3)
## 6 Anaheratail(asot_top_1000_df)## Rank Artist Song
## 995 995 Elevven One Last Time
## 996 996 Late Night Alumni Empty Streets (Lumïsade Balearic Mix)
## 997 997 ATN Miss A Day
## 998 998 Nic Chagall Monday Bar
## 999 999 Mark Otten So Serene (Menno de Jong Fierce Dub)
## 1000 1000 Robert Nickson Circles
The count() function from dplyr can be used to obtain counts.
# Top 20 Most Frequent Artists in ASOT Top 1000
# Armin van Buuren, Above & Beyond, GAIA, Giuesppe Ottaviani, Ørjan Nilsen, Aly & Fila, Gareth Emery
asot_top_1000_df %>%
count(Artist) %>%
arrange(desc(n)) %>%
head(20)## Artist n
## 1 Armin van Buuren 31
## 2 Above & Beyond 13
## 3 GAIA 12
## 4 Above & Beyond feat. Zoë Johnston 10
## 5 Giuseppe Ottaviani 10
## 6 Ørjan Nilsen 10
## 7 Above & Beyond pres. OceanLab 9
## 8 Andrew Rayel 9
## 9 Ferry Corsten 9
## 10 Above & Beyond feat. Richard Bedford 8
## 11 Gouryella 8
## 12 Aly & Fila 7
## 13 Gareth Emery 7
## 14 Tiësto 7
## 15 Jorn van Deynhoven 6
## 16 Solarstone 6
## 17 Super8 & Tab 6
## 18 W&W 6
## 19 Cosmic Gate 5
## 20 4 Strings 4# Save table as .csv file:
write.csv(asot_top_1000_df, file = 'ASOT_Top1000_Tracks.csv', row.names = FALSE)