Compare commits
9
Commits
412b349007
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a2663b2e90 | ||
|
|
ab0fa33172 | ||
|
|
83cdcc279b | ||
|
|
1e9c9ba651 | ||
|
|
efddf043e3 | ||
|
|
c374e8cd8e | ||
|
|
b7166d1578 | ||
|
|
896ba1b3b0 | ||
|
|
ecb5677703 |
@@ -7,6 +7,7 @@ export(join_speaker)
|
||||
export(party_colors)
|
||||
export(read_all)
|
||||
export(read_from_csv)
|
||||
export(read_from_csv_or_fetch)
|
||||
export(repair)
|
||||
export(word_usage_by_date)
|
||||
export(write_to_csv)
|
||||
|
||||
@@ -271,3 +271,23 @@ read_from_csv <- function(path="inst/csv/") {
|
||||
is_valid_res(res)
|
||||
res
|
||||
}
|
||||
|
||||
#' Read records from csv or fetch
|
||||
#'
|
||||
#' @param path base directory where csv files are expected under path/csv
|
||||
#' and possibly records fetched and stored under path/records
|
||||
#'
|
||||
#' @export
|
||||
read_from_csv_or_fetch <- function(path="inst/") {
|
||||
path <- make_directory_path(path)
|
||||
res <- tryCatch(read_from_csv(str_c(path, "csv/")),
|
||||
error = function(c) NULL)
|
||||
if (!is.null(res)) return(res)
|
||||
|
||||
fetch_all(str_c(path, "records/"), create=T)
|
||||
read_all(str_c(path, "records/")) %>%
|
||||
repair() ->
|
||||
res
|
||||
write_to_csv(res, str_c(path, "csv/"), create=T)
|
||||
res
|
||||
}
|
||||
|
||||
@@ -9,26 +9,34 @@ Using the `remotes` package, this is easily installed via:
|
||||
```r
|
||||
remotes::install_url("https://git.flavigny.de/christian/hateimparlament/archive/master.zip")
|
||||
```
|
||||
Since the fetching and reading is very slow and depends on an internet connection, all vignettes
|
||||
use `read_from_csv` to read already parsed tibbles from `.csv` files.
|
||||
If you want to build the vignettes, pass `build_vignettes = TRUE`. This takes a long time and
|
||||
fails sometimes, if bundestag.de times out, since
|
||||
in the beginning the necessary records are neither fetched nor parsed.
|
||||
|
||||
That's why, if you want to build the vignettes yourself, you need to
|
||||
download the source code, e.g. on Linux
|
||||
## Install with vignettes
|
||||
|
||||
An alternative for building
|
||||
the vignettes is to clone the repository and build the vignettes manually, e.g. on Linux
|
||||
```
|
||||
git clone https://git.flavigny.de/christian/hateimparlament
|
||||
cd hateimparlament
|
||||
```
|
||||
then start `R` and do
|
||||
Then open a `R` shell and do
|
||||
```r
|
||||
devtools::load_all() # load package
|
||||
devtools::wd() # set working directory
|
||||
```
|
||||
Then fetch all records, read them and write the parsed tibbles to csv files.
|
||||
```r
|
||||
devtools::load_all()
|
||||
fetch_all(create = TRUE)
|
||||
read_all() %>% repair() -> res
|
||||
write_to_csv(res, create = TRUE)
|
||||
```
|
||||
Then finally, do:
|
||||
Now you can install the package with vignettes by using
|
||||
```r
|
||||
devtools::install(build_vignettes = TRUE)
|
||||
```
|
||||
This makes all vignettes available via `browseVignettes()`.
|
||||
|
||||
# Features
|
||||
|
||||
|
||||
@@ -24,5 +24,5 @@
|
||||
In der zweiten Hälfte des Projekts kümmerten wir uns dann um die Analyse der Daten und stellten unsere Ergebnisse in Vignetten da und erzeugten Dokumentationen für alle Funktionen, die für den Nutzer wichtig sind.
|
||||
\newpage
|
||||
\section*{Meine Beteiligung}
|
||||
|
||||
\input{meine_beteiligung.tex}
|
||||
\end{document}
|
||||
@@ -0,0 +1,15 @@
|
||||
% Generated by roxygen2: do not edit by hand
|
||||
% Please edit documentation in R/parse.R
|
||||
\name{read_from_csv_or_fetch}
|
||||
\alias{read_from_csv_or_fetch}
|
||||
\title{Read records from csv or fetch}
|
||||
\usage{
|
||||
read_from_csv_or_fetch(path = "inst/")
|
||||
}
|
||||
\arguments{
|
||||
\item{path}{base directory where csv files are expected under path/csv
|
||||
and possibly records fetched and stored under path/records}
|
||||
}
|
||||
\description{
|
||||
Read records from csv or fetch
|
||||
}
|
||||
@@ -34,9 +34,10 @@ read_all("../inst/records/") %>% repair() -> res
|
||||
```
|
||||
We also used `repair` to fix a bunch of formatting issues in the records.
|
||||
|
||||
For development purposes, we load the tables from csv files.
|
||||
For development purposes, we only fetch records if they are not already
|
||||
stored as csv files:
|
||||
```{r}
|
||||
res <- read_from_csv('../inst/csv/')
|
||||
res <- read_from_csv_or_fetch('../inst/')
|
||||
```
|
||||
|
||||
## Analysis
|
||||
|
||||
@@ -35,10 +35,13 @@ read_all("../records/") %>% repair() -> res
|
||||
```
|
||||
We also used `repair` to fix a bunch of formatting issues in the records.
|
||||
|
||||
For development purposes, we load the tables from csv files.
|
||||
For development purposes, we only fetch records if they are not already
|
||||
stored as csv files:
|
||||
```{r}
|
||||
res <- read_from_csv('../inst/csv/')
|
||||
res <- read_from_csv_or_fetch('../inst/')
|
||||
```
|
||||
|
||||
|
||||
and unpack our tibbles
|
||||
```{r}
|
||||
comments <- res$comments
|
||||
|
||||
@@ -34,9 +34,10 @@ read_all("../inst/records/") %>% repair() -> res
|
||||
```
|
||||
We also used `repair` to fix a bunch of formatting issues in the records.
|
||||
|
||||
For development purposes, we load the tables from csv files.
|
||||
For development purposes, we only fetch records if they are not already
|
||||
stored as csv files:
|
||||
```{r}
|
||||
res <- read_from_csv('../inst/csv/')
|
||||
res <- read_from_csv_or_fetch('../inst/')
|
||||
```
|
||||
|
||||
## Analysis
|
||||
|
||||
@@ -38,9 +38,10 @@ talks <- res$talks
|
||||
We also used `repair` to fix a bunch of formatting issues in the records and unpacked
|
||||
the result into more descriptive variables.
|
||||
|
||||
For development purposes, we load the tables from csv files.
|
||||
For development purposes, we only fetch records if they are not already
|
||||
stored as csv files:
|
||||
```{r}
|
||||
tables <- read_from_csv('../inst/csv/')
|
||||
tables <- read_from_csv_or_fetch('../inst/')
|
||||
|
||||
comments <- tables$comments
|
||||
speeches <- tables$speeches
|
||||
|
||||
@@ -34,9 +34,10 @@ read_all("../inst/records/") %>% repair() -> res
|
||||
```
|
||||
We also used `repair` to fix a bunch of formatting issues in the records.
|
||||
|
||||
For development purposes, we load the tables from csv files.
|
||||
For development purposes, we only fetch records if they are not already
|
||||
stored as csv files:
|
||||
```{r}
|
||||
res <- read_from_csv('../inst/csv/')
|
||||
res <- read_from_csv_or_fetch('../inst/')
|
||||
```
|
||||
|
||||
## Analysis
|
||||
|
||||
Reference in New Issue
Block a user