Yahoo unhappy about crumbs

[eddelbuettel]

Description

getQuote() is no longer working.

Expected behavior

getQuote("SPY") etc worked but no longer does

Minimal, reproducible example

> quantmod::getQuote("SPY")
Error in .yahooSession(TRUE) : unable to get yahoo crumb
> quantmod:::.yahooSession()
Error in .yahooSession(TRUE) : unable to get yahoo crumb
> quantmod:::.yahooSession(TRUE)
Error in quantmod:::.yahooSession(TRUE) : unable to get yahoo crumb
> 

Session Info

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 23.04

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.11.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.11.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8      
 [8] LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/Chicago
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] quantmod_0.4.24 TTR_0.24.3      xts_0.13.1      zoo_1.8-12     

loaded via a namespace (and not attached):
[1] compiler_4.3.1 tools_4.3.1    parallel_4.3.1 colorout_1.2-2 curl_5.0.1     grid_4.3.1     jsonlite_1.8.7 lattice_0.21-8 fortunes_1.5-4
> 

[ethanbsmith]

can you try this branch: remotes::install_github("ethanbsmith/quantmod@fix_404_handle_getquote_gdpr_errors")

[eddelbuettel]

Works like a charm after a quick (command-line, via littler)

  installGithub.r ethanbsmith/quantmod@fix_404_handle_getquote_gdpr_errors

(which of course wraps the remotes call). All good so suggest to merge.

[eddelbuettel]

And just for completeness as I use this in a loop storing in Redis (details on my blog / in package dang / happy to expand): Behavior change apparently commenced at 23:00h Central yesterday or 00:00h Eastern.

[rjvelasquezm]

I can confirm this fix is working for me as well.

[joshuaulrich]

This seems to be caused by a missing "User-Agent" in the current request header. Ethan's patch added a user-agent in addition to improving the error message.

And it's not good enough for the user-agent to be set to something. It needs to be a value the server considers valid, like it is in Ethan's patch. I tried setting it to "R (4.3.1, x86_64-pc-linux-gnu, x86_64, linux-gnu)" but the request still failed.

[ethanbsmith]

this is the second change yahoo has made in the last few months that treats scripting differently than actual browsers. i fear this api is not long for this world. just my .02