The finRes suite is organised around the data-science pipeline where preprocessing, including data collection and wrangling, plays a major role. finRes addresses the issue in two complementary packages that work in conjunction with most of the finRes dataset packages.
On the one hand the pullit package provides tools for data collection from Bloomberg. It returns clean and tidy, ready-to-use, data objects for other packages further down the pipeline to work with. On the other hand the storethat package works in concert with fewISOs and GICS to help store the data retrieved for off-Bloomberg consumption in R.
Both pullit and storethat work in tandem with the BBGsymbols package. The latter plays a central role in finRes where it helps pullit in interacting with Bloomberg via the interface provided by the Rblpapi package (Armstrong, Eddelbuettel, and Laing 2021) and storethat in storing the data retrieved.
pullit together with storethat are the two workhorses for data ETL (extract, transform, load) in the finRes suite. Using carefully selected Bloomberg datafields from BBGsymbols in tandem with Armstrong, Eddelbuettel, and Laing (2021)’s Bloomberg interface pullit provides the R user with easy access to Bloomberg financial data for a number of financial instruments including, at the time of writing, equity and equity-like securities, funds with the category encompassing any money-managing entity, indexes, as well as futures series & term structure individual contracts.
For each instrument category, pullit allows data retrieval for a corresponding category specific set of data types. An active Bloomberg connection is required to retrieve Bloomberg financial data using pullit.
Equity data comes in three major categories in pullit, market, book and info. For a given corporation market data records stock market activity while book data records business activity in the form of financial statements commonly referred to as ‘books’ and info refers to qualitative information. See the fields dataset in BBGsymbols for a detailed list of the Bloomberg datafields available.
Retrieve market data from Bloomberg for British Petroleum (BP/ LN Equity), the Weir group (WEIR LN Equity) and Apple (AAPL US Equity) with:
equity_tickers <- c('BP/ LN Equity', 'WEIR LN Equity', 'AAPL US Equity')
equity_market <- pull_equity_market(source = "Bloomberg", equity_tickers, start, end, verbose = F)
equity_market#> S4 object of class EquityMarket
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()For financial statements data BBGsymbols replicates the Bloomberg ‘financial analysis’ monitor (FA <GO>). At the time of writing, this includes balance sheet, cash flow statement, income statement as well as ‘key stats’ that gathers broad summary figures and ‘ratios’ that includes popular financial ratios.
Retrieve the corresponding data for the abovementioned corporations with:
equity_BS <- pull_equity_book(source = "Bloomberg", book = "balance sheet", equity_tickers,
start, end, verbose = F)
equity_BS#> S4 object of class EquityBS
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()
equity_CF <- pull_equity_book(source = "Bloomberg", book = "cash flow statement", equity_tickers,
start, end, verbose = F)
equity_CF#> S4 object of class EquityCF
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()
equity_IS <- pull_equity_book(source = "Bloomberg", book = "income statement", equity_tickers,
start, end, verbose = F)
equity_IS#> S4 object of class EquityIS
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()
equity_KS <- pull_equity_book(source = "Bloomberg", book = "key stats", equity_tickers,
start, end, verbose = F)
equity_KS#> S4 object of class EquityKS
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()
equity_R <- pull_equity_book(source = "Bloomberg", book = "ratios", equity_tickers,
start, end, verbose = F)
equity_R#> S4 object of class EquityRatios
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()‘info’ encompasses a range of contemporaneous qualitative information including, but not limited to, firm’s name, security type, exchange where the security trades, country of incorporation, etc.
Retrieve the corresponding data for the abovementioned corporations with:
equity_info <- pull_equity_info(source = "Bloomberg", equity_tickers, verbose = F)
equity_info#> S4 object of class EquityInfo
#>
#> Slots inlude
#> info: access with get_info()
#> fields: access with get_fields()
#> call: access with get_call()Fund data only comes in two categories, market and info. For a given fund market data records stock market activity while info data records contemporaneous qualitative information that includes a wide range of fund characteristics. Market historical data as well as contemporaneous qualitative data not only allow for an in depth assessment of the fund performance but potentially also the mapping of the performance to fund characteristics. See the fields dataset in BBGsymbols for a detailed list of the Bloomberg datafields available.
Retrieve market data from Bloomberg for the SPDR S&P 500 ETF Trust (SPY US Equity):
fund_tickers <- "SPY US Equity"
fund_market <- pull_fund_market(source = "Bloomberg", fund_tickers, start, end, verbose = F)
fund_market#> S4 object of class FundMarket
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()Retrieve the corresponding qualitative data for the abovementioned funds with:
fund_info <- pull_fund_info(source = "Bloomberg", fund_tickers, verbose = F)
fund_info#> S4 object of class FundInfo
#>
#> Slots inlude
#> info: access with get_info()
#> fields: access with get_fields()
#> call: access with get_call()Futures data comes in three major categories in pullit, market, CFTC and info. For a given futures series market data records futures market price activity while CFTC data records market positions. ‘info’ on the other hand records contemporaneous qualitative information for the corresponding futures series, including but not limited to series name, term structure length, contract size, contract unit, etc. See the fields dataset in BBGsymbols for a detailed list of the Bloomberg datafields available.
pullit helps to retrieve both term structure as well as aggregated market data for futures. Term structure data records market activity for individual futures term structure contracts while aggregated data records market activity measures that are aggregated over the whole term structure for a particular futures series.
Retrieve market data from Bloomberg for the five first term structure contracts on the Corn - #2-yellow (C A Comdty) and silver (SIA Comdty) futures series where the futures chains are constructed by rolling on liquidity (with active contract: “A”) with no roll adjustment (none: “N”) using:
futures_tickers <- c("C A Comdty", "SIA Comdty")
futures_TS <- pull_futures_market(source = "Bloomberg", type = "term structure",
active_contract_tickers = futures_tickers,
start, end, TS_positions = 1L:5L, roll_type = "A", roll_days = 0L,
roll_months = 0L, roll_adjustment = "N", verbose = F)
futures_TS#> S4 object of class FuturesTS
#>
#> Slots inlude
#> active_contract_tickers: access with get_active_contract_tickers()
#> term_structure_tickers: access with get_term_structure_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()Retrieve the corresponding aggregated futures market data with:
futures_agg <- pull_futures_market(source = "Bloomberg", type = "aggregate",
active_contract_tickers = futures_tickers, start, end, verbose = F)
futures_agg#> S4 object of class FuturesAggregate
#>
#> Slots inlude
#> active_contract_tickers: access with get_active_contract_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()pullit helps to retrieve CFTC futures market position data from Bloomberg. The Commodity Futures Trading Commission (CFTC) publishes the Commitments of Traders (COT) reports to help the public understand market dynamics. Specifically, the COT reports provide a breakdown of each Tuesday’s open interest for futures and options on futures markets in which 20 or more traders hold positions equal to or above the reporting levels established by the CFTC. See the fields and tickers_CFTC datasets in BBGsymbols for details.
Retrieve the corresponding futures market position data with:
futures_CFTC <- pull_futures_CFTC(source = "Bloomberg", active_contract_tickers = futures_tickers,
start, end, verbose = F)
futures_CFTC#> S4 object of class FuturesCFTC
#>
#> Slots inlude
#> active_contract_tickers: access with get_active_contract_tickers()
#> cftc_tickers: access with get_CFTC_tickers()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()‘info’ encompasses a range of contemporaneous qualitative information on the underlying futures series including, but not limited to, name for the series’ underlying, trading exchange, term structure length, contract size, etc.
Retrieve the corresponding data for the abovementioned futures series with:
futures_info <- pull_futures_info(source = "Bloomberg", futures_tickers, verbose = F)
futures_info#> S4 object of class FuturesInfo
#>
#> Slots inlude
#> info: access with get_info()
#> fields: access with get_fields()
#> call: access with get_call()The functions above return objects that not only carry the retrieved financial data but also complementary information that can be accessed using bespoke accessor methods. Historical data functions for example return objects that carry a ‘tickers’ dataframe that indicates the tickers for which some data have been found, a ‘fields’ dataframe that indicates the data fields for which data has been found, a ‘data’ dataframe that hosts the retrieved data as well as a character vector hosting the original call to the function. A get_periods() method complements the accessor methods by indicating the start and end dates between which data have been found for each ticker and data field:
futures_TS
#> S4 object of class FuturesTS
#>
#> Slots inlude
#> active_contract_tickers: access with get_active_contract_tickers()
#> term_structure_tickers: access with get_term_structure_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()Access each slot using the appropriate accessor:
tickers <- get_active_contract_tickers(futures_TS)
tickers
#> active contract ticker
#> 1: C A Comdty
#> 2: SIA Comdty
fields <- get_fields(futures_TS)
head(fields)
#> active contract ticker ticker instrument book type
#> 1: C A Comdty C 1 A:00_0_N Comdty futures market term structure
#> 2: C A Comdty C 1 A:00_0_N Comdty futures market term structure
#> 3: C A Comdty C 1 A:00_0_N Comdty futures market term structure
#> 4: C A Comdty C 1 A:00_0_N Comdty futures market term structure
#> 5: C A Comdty C 1 A:00_0_N Comdty futures market term structure
#> 6: C A Comdty C 1 A:00_0_N Comdty futures market term structure
#> symbol
#> 1: OPEN_INT
#> 2: PX_ASK
#> 3: PX_BID
#> 4: PX_HIGH
#> 5: PX_LAST
#> 6: PX_LOW
data <- pullit::get_data(futures_TS)
data
#> ticker field date value
#> 1: C 1 A:00_0_N Comdty OPEN_INT 2016-01-04 727298
#> 2: C 1 A:00_0_N Comdty OPEN_INT 2016-01-05 730600
#> 3: C 1 A:00_0_N Comdty OPEN_INT 2016-01-06 727995
#> 4: C 1 A:00_0_N Comdty OPEN_INT 2016-01-07 727244
#> 5: C 1 A:00_0_N Comdty OPEN_INT 2016-01-08 719406
#> ---
#> 41598: SI5 A:00_0_N Comdty PX_VOLUME 2017-12-22 123
#> 41599: SI5 A:00_0_N Comdty PX_VOLUME 2017-12-26 58
#> 41600: SI5 A:00_0_N Comdty PX_VOLUME 2017-12-27 71
#> 41601: SI5 A:00_0_N Comdty PX_VOLUME 2017-12-28 91
#> 41602: SI5 A:00_0_N Comdty PX_VOLUME 2017-12-29 151
call <- pullit::get_call(futures_TS)
call
#> storethat_futures_TS(file = file, active_contract_tickers = active_contract_tickers,
#> start = start, end = end, TS_positions = TS_positions, roll_type = roll_type,
#> roll_days = roll_days, roll_months = roll_months, roll_adjustment = roll_adjustment,
#> verbose = verbose)All the objects above can be stored in a bespoke database for later off-Bloomberg consumption. The storethat package makes the process seamless by providing befitted storing methods:
library(storethat)
db_create()
path = "~/storethat.sqlite"
db_store(object = futures_TS, file = path, verbose = F)
db_store(object = fund_market, file = path, verbose = F)All the functions described above can be used to query data from a storethat database by setting the source parameter to “storethat” and providing the path to the database as a string in the file parameter. If omitted a pop-up window allows to browse the file system and select the database file. The other function parameters are identical for both data sources and the returned objects are identical in all aspects other than the source of the data they contain:
equity_market <- pull_equity_market(
source = "storethat", equity_tickers, start, end, verbose = F, file = path
)
equity_market
#> S4 object of class EquityMarket
#>
#> Slots inlude
#> tickers: access with get_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()Updating a storethat database is equally straightforward with pullit. Update the equity content of the database with:
storethat_update(instrument = "equity", file = path)The function pulls data from Bloomberg for all the equity tickers and fields already present in the storethat database and updates the corresponding series up to the current system date.
Narrower updates are also allowed:
storethat_update(instrument = "equity", book = "market")The plotit package, also part of the finRes suite, provides plot methods for some pullit data objects including, at the time of writing, futures term structure (FuturesTS) and fund market (FundMarket) objects.
Plot historical fund performance with:
plot(object = fund_market, ticker = "SPY US Equity")Armstrong, Whit, Dirk Eddelbuettel, and John Laing. 2021. Rblpapi: R Interface to ’Bloomberg’. https://CRAN.R-project.org/package=Rblpapi.