180 Commits (b406304c80f7a013c457fb9967dc6ddd982b2355)

 

All Branches   

Search

Author	SHA1	Message	Date
Your Name	b406304c80	actorizer: Removed nested parallelization function ... query_gen_actors: Integrated startDate and endDate for parties, changed party exception method from abbreviation only to both full names and abbreviations for NL and BE	4 years ago
Your Name	5de4e1488c	estimator, modelizer, preproc: Removed experimental we-vector support, and disabled inefficiently implemented preproc.R	4 years ago
Your Name	77eb51a1bf	actorizer: totally revamped way of finding actors ... elasticizer: updated dump handling to create a dump for every batch, instead of one big file at the end out_parser: streamlined code query_gen_actors: only include relevant fields ud_update: changed function parameters to work with elasticizer dump function	4 years ago
Your Name	0e593075ee	query_gen_actors: only retrieve ud field from source	4 years ago
Your Name	6eb405f8bd	merger: selecting only relevant columns	4 years ago
Your Name	38ff4dcbf0	ud_update: small fix to file naming	4 years ago
Your Name	4b4d860235	class_update: remove dfm_gen multicore option ... dfm_gen: remove multicore, update merger() code elasticizer: changed filenaming scheme for dump option merger: Fixed bug where an NA lemma would cause the entire document to become NA. Now the NA lemmas are filtered out before merging ud_update: removed parallel processing, changed script to save bulk updates in .Rds files instead of sending them straight away	4 years ago
Your Name	5d99ec9509	elasticizer: added option to dump data frames to rds files ... out_parser: changed to single core, due to performance increase sentencizer: corrected documentation for sent_dict parameter	4 years ago
Your Name	aa6587b204	dupe_detect: fix for quotation marks	4 years ago
Your Name	2a220ded5d	dupe_detect: fix to query string for multi-word doctype names	4 years ago
Your Name	5bd36dcb44	dupe_detect: Changed query from json to query_string style, and added filter for already detected duplicates ... cv_generator: Changed code to use a generic vector of true values to draw the conditional random sample, instead of dfm/docvars specifically	4 years ago
Your Name	e499d70671	actor_merger: added ungroup() calls at the start and end of function, to speed up processing ... sentencizer: added ungroup() call at the end of the function to speed up processing	4 years ago
Your Name	8634d549a3	sentencizer: updates to collect sentence word counts and number of sentences also when no sent_dict is provided	4 years ago
Your Name	61e0581595	actor_merger: removed debug line	4 years ago
Your Name	11bf71c7dd	fixes for removal of actor_fetcher function	4 years ago
Your Name	f022312485	actor_merger: added function for generating actor-document data frames ... actor_fetcher: removed from package other: major update to documentation	4 years ago
Your Name	4e867214dd	sentencizer: commented code	4 years ago
Your Name	ec8afc4990	sentencizer: fixed actorsDetail coding error	4 years ago
Your Name	9ccfd2952e	sentencizer: minor updates	4 years ago
Your Name	98325bde8f	sentencizer: added new function for sentiment coding and actor collection	4 years ago
Your Name	7f958bbc11	actor_fetcher: small fixes	4 years ago
Your Name	8eedec8bb5	actor_fetcher: added option for using dictionaries with just lemmas, besides the option of using lemma_upos dictionaries	4 years ago
Your Name	057d225a7a	actor_fetcher: Allow generation of actor df containing only specified actor ids and aggregations	4 years ago
Your Name	9eae486a80	separated data preprocessing routines ... class_update: check if there are idf values associated with model, before applying weights estimator: make use of preproc() function for data preprocessing preproc: function containing all logic with regards to text data preprocessing and weighting	5 years ago
Your Name	a3b6e19646	revised modeling pipeline: ... cv_generator: generate folds for nested cv dfm_gen: added optional lowercasing parameter estimator: estimate model and performance based on parameters feat_select: select features based on textstat_keyness metric_gen: convert output from estimator to model performance metrics modelizer: updated for new pipeline modelizer_old: old model pipeline out_parser: now correctly exported	5 years ago
Your Name	e76a914dd2	actor_fetcher: Updated to tidyr 1.0.0, no longer using preserve, slightly different approach to keeping ids_list, and not removing actorsDetail anymore because it does not exist	5 years ago
Your Name	a01a53f105	class_update: added cores parameter for multicore processing of sources when using lemmas	5 years ago
Your Name	d9f936c566	modelizer: tf-idf application updated, final model now also includes idf values from training set, explicitly setting positive category in binary classification for confusion matrices, minor code fixes ... dfm_gen: added old junk codes for recoding, and removed deprecated ngrams parameter from dfm function class_update: removed dfm_words parameter, which is replaced by the force = T parameter in predict(), training/model idf is now applied to unseen data DESCRIPTION: added quanteda.textmodels as new dependency, since these have been separated from base quanteda 2.0.0 onwards	5 years ago
Erik de Vries	06bfec71bc	lemma_writer: unlist lemmas before writing	5 years ago
Erik de Vries	a83ee5dfd0	lemma_writer: update to write lemma instead of full document text	5 years ago
Erik de Vries	e594185719	dfm_gen: set default cores to 1	5 years ago
Erik de Vries	889e7e92af	lemma_writer: updated to provide support for writing raw documents to individual files using utf-8 encoding	5 years ago
Erik de Vries	115297f597	actor_aggregation,aggregator,aggregator_elastic: moved out of package directory to Old ... actor_fetcher: moved sentiment validation code block	5 years ago
Erik de Vries	3fcbbd1f1f	actor_fetch: fixed error where source.ud would not exist	5 years ago
Erik de Vries	674ef09e10	query_gen_actors: added junior minister check to if statement	5 years ago
Erik de Vries	853c117daf	actor_fetcher: change in code to keep original actorid lists in output ... query_gen_actors: added code for junior ministers in BE and NL	5 years ago
Erik de Vries	bf3d11ffe0	query_gen_actors: various bugfixes and changes	5 years ago
Erik de Vries	99af1427f0	query_gen_actors: fixed scandinavian query generation	5 years ago
Erik de Vries	e49a4ae93e	query_gen_actors: fixed problem with too many brackets in query	5 years ago
Erik de Vries	060751237b	actorizer, out_parser: switched from mclapply to future_lapply and removed windows-specific code from out_parser ... query_gen_actors: rewritten minister queries to only use proximity queries	5 years ago
Erik de Vries	d0601d2aa7	actor_fetcher: added minimum verbosity to identify cases in which an actor is present without a party mention	5 years ago
Erik de Vries	82ef165c5f	actor_fetcher: quick fix	5 years ago
Erik de Vries	9e433ecf9e	actor_fetcher: added handling of exception where all actorsids related to a party are individual actors	5 years ago
Erik de Vries	526270900c	actor_fetcher: integrated party merging into actor_fetcher in what hopefully is the most efficient way	5 years ago
Erik de Vries	84df9658ff	actor_fetcher: added lemma output when validating, to detect most problematic lemmas	5 years ago
Erik de Vries	499ee74f0d	actor_fetcher: fixed code error	5 years ago
Erik de Vries	a3e8dcf96e	actor_fetcher: switched from binary word sentiment scores to proximity scores (cosine similarity)	5 years ago
Erik de Vries	6f5ace8c52	actor_fetcher: elasticizer batch function to fetch actorsDetail fields from all relevant documents	5 years ago
Erik de Vries	edd4b785a5	actor_aggregation: updated to use future package for parallel processing as beta test for switching all parallel processing to future. Also disabled some of the aggregator output to save computation time	5 years ago
Erik de Vries	f8bc53006d	actor_aggregation: added sentiment analysis support for generating aggregations	5 years ago