diff --git a/git.md b/git.md index e87b21b..91855b5 100644 --- a/git.md +++ b/git.md @@ -3,18 +3,19 @@ Branches ============================================ -### Adding Branches ### +### Adding Branches + * local: `git checkout -b ` * remote: `git push --set-upstream ` * track remote: `git checkout --track /` -### Deleting Branches ### +### Deleting Branches * local: `git branch -d ` * remote: `git push -d ` * realize remote deletes: `git remote prune ` -### Non-Standard Activities ### +### Non-Standard Activities * merge only a single file into another branch `git checkout -- ` * delete from repo and file system `git rm ` @@ -24,4 +25,4 @@ Config ============================================= * set line ending behaviour `git config --global core.autocrlf true` -* store credentials `git config credential.helper store` or `git config credential.helper cache` \ No newline at end of file +* store credentials `git config credential.helper store` or `git config credential.helper cache` diff --git a/mastodon.md b/mastodon.md index 29186bf..daefb64 100644 --- a/mastodon.md +++ b/mastodon.md @@ -1,146 +1,4 @@ -issue with mastodon-streaming service. +Mastodon is ruby app. +you need a ruby environment manager just like node, need to be aware of this as ruby environment upgrades in combination with mastodon upgrades have not worked well. -``` -Jan 19 23:05:26 r710 node[17762]: /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:34 -Jan 19 23:05:26 r710 node[17762]: throw e.message = e.message + " check './node_modules/@clusterws/cws/build_log.txt' for post install build logs", -Jan 19 23:05:26 r710 node[17762]: ^ -Jan 19 23:05:26 r710 node[17762]: Error: The module '/home/mastodon/live/node_modules/@clusterws/cws/dist/cws_linux_79.node' -Jan 19 23:05:26 r710 node[17762]: was compiled against a different Node.js version using -Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 72. This version of Node.js requires -Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 79. Please try re-compiling or re-installing -Jan 19 23:05:26 r710 node[17762]: the module (for instance, using `npm rebuild` or `npm install`). check './node_modules/@clusterws/cws/build_log.txt' for post install build logs -Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..node (internal/modules/cjs/loader.js:1194:18) -Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32) -Jan 19 23:05:26 r710 node[17762]: at Function.Module._load (internal/modules/cjs/loader.js:892:14) -Jan 19 23:05:26 r710 node[17762]: at Module.require (internal/modules/cjs/loader.js:1033:19) -Jan 19 23:05:26 r710 node[17762]: at require (internal/modules/cjs/helpers.js:72:18) -Jan 19 23:05:26 r710 node[17762]: at /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:32:16 -Jan 19 23:05:26 r710 node[17762]: at Object. (/home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:37:3) -Jan 19 23:05:26 r710 node[17762]: at Module._compile (internal/modules/cjs/loader.js:1144:30) -Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:1164:10) -Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32) -``` -per [node webiste](https://nodejs.org/en/download/releases/) node module version 72 NodeJS v 12.14.1 and npm version 6.13.4 - -live/streaming hold the top level code. -if I try to run `node index.js` and hardcode the REDIS password, end up with postgres authentication error due to the connection module not supporting SCRAM-SHA-256 - -changed that back: real issue was that node was reverting to the latest version instead of LTS when starting the service since NVM is only per session. -apt n module makes a permanent version change, used that instead - -so trust fixes a manual run of index.js, but the having that available on port 4000 doesn't help the search function. - - -now it is clear that search issue doesn't have anything to do with resolved streaming API service. -shoudl try service maybe? - -found this in the issues: -https://github.com/tootsuite/mastodon/issues/5765 - - -This issue notes a web domain setting, grep of mastodon directory gives: -``` -./live/config/initializers/1_hosts.rb:web_host = ENV.fetch('WEB_DOMAIN') { host } -./live/lib/mastodon/premailer_webpack_strategy.rb: asset_host = ENV['CDN_HOST'] || ENV['WEB_DOMAIN'] || ENV['LOCAL_DOMAIN'] -./live/.env.nanobox:# WEB_DOMAIN=mastodon.example.com -./live/.env.nanobox:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN -./live/.env.nanobox:# if WEB_DOMAIN is not set. For example, the server may have the -./live/.env.nanobox:# The attachment host must allow cross origin request from WEB_DOMAIN or -./live/.env.nanobox:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the -./live/.env.production.sample:# WEB_DOMAIN=mastodon.example.com -./live/.env.production.sample:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN -./live/.env.production.sample:# if WEB_DOMAIN is not set. For example, the server may have the -./live/.env.production.sample:# The attachment host must allow cross origin request from WEB_DOMAIN or -./live/.env.production.sample:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the -``` - -notes in env.production.example say not to set `WEB_DOMAIN` - - - -posted a question on [discourse](https://discourse.joinmastodon.org/t/search-return-404/2490) - - -Issue with searching accounts on other instances - -syslog: -``` -Jan 30 20:35:50 r710 bundle[7118]: [a80db716-134c-4ca2-9c18-785675ec716d] method=GET path=/api/v2/search format=html controller=Api::V2::SearchController action=index status=404 duration=618.40 view=0.41 db=3.85 -``` - -nginx access.log -``` -[30/Jan/2020:20:35:50 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 54 "https://hptrow.me/" "Mozilla/5.0 (Android 8.1.0; Mobile; rv:68.0) Gecko/68.0 Firefox/68.0" -``` - -IRC convo: -``` -18:05 < pinkieval> what version is your instance? -18:13 < ptbridge> 3.0.1 -18:14 < pinkieval> mm -18:14 < ptbridge> yeah -18:15 < pinkieval> so apparently it recognized the URL (because of "controller=Api::V2::SearchController") but somehow still returns a 404 -18:15 < pinkieval> maybe ?q= is missing? -18:17 < ptbridge> nginx access.log -> "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" -18:17 < ptbridge> missed the 404 "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 -18:19 < pinkieval> https://mastodon.social/api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 is not a 404 -18:19 < pinkieval> what domain is your instance? -18:19 < ptbridge> hptrow.me -18:19 < pinkieval> I'm not getting a 404 either -18:20 < ptbridge> i just set it up and was messing around, and noticed search didn't seem to hook up outside of my instance -18:20 < pinkieval> I'm getting a 401 -18:21 < ptbridge> ¯\_(ツ)_/¯ -18:22 < ptbridge> thanks for checking -18:23 < pinkieval> how are you getting that 404? -18:24 < pinkieval> is it because you're using an API token? -18:26 < ptbridge> i haven't tried that, i just tried to do the search on the webpage and it didn't work so I checked my logs and that's what i saw -18:26 < pinkieval> uh -18:27 < ptbridge> like literally go up to the search box and type tusky@mastodon.social and nothing come back -18:27 < ptbridge> comes* -18:28 < pinkieval> I just created an account on your instance so I can test, can you approve it? -18:29 < pinkieval> I'm val -18:30 < ptbridge> done -18:31 < pinkieval> it's working for me -18:31 < ptbridge> wuh -18:31 < ptbridge> lol -18:31 < ptbridge> hang on let me check the logs -18:32 < ptbridge> sure enough -> "GET /api/v2/search?q=mastodon&resolve=true&limit=5 HTTP/2.0" 200 -18:33 < ptbridge> try searchign for an account on another instance -18:34 < pinkieval> oh right, 404 now -18:34 < pinkieval> that's weird -18:35 < ptbridge> yeah -> 51.159.34.167 - - [31/Jan/2020:18:34:12 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 -18:35 < ptbridge> oh well, thanks so much for checking that out! -18:36 < pinkieval> do you have Elasticsearch configured? -18:37 < ptbridge> yeah I have no idea, i saw mention of that, but don't see that as a service running -18:37 < pinkieval> so no -18:38 < pinkieval> so that's not the source of the issue -18:40 < pinkieval> ah! error response is: hptrow.me -18:40 < pinkieval> meh -18:40 < pinkieval> ah! error response is: {"error":"Record not found"} -18:41 < pinkieval> that's partially helpful -18:41 < ptbridge> does the API itself have a log it is dumping to somewhere? -18:42 < ptbridge> I just have a log for the result of the call to the API -18:43 < pinkieval> afaik, no -18:45 < ptbridge> It probably something dumb I didn't setup right with nginx or something -18:45 < ptbridge> alright, thanks so much -18:49 < pinkieval> your instance does reach mine when I make these requests, though: 174.104.140.112 "oc.todon.fr" - [31/Jan/2020:23:38:42 +0000] "GET /.well-known/webfinger?resource=acct:val@oc.todon.fr HTTP/1.1" 200 1649 "-" "http.rb/3.3.0 (Mastodon/3.0.1; - +https://hptrow.me/)" -18:52 -!- Pythayr [~pither195@38.132.120.242] has quit [Read error: Connection reset by peer] -18:54 < ptbridge> how is that request generated? -18:56 < pinkieval> when I search for @val@oc.todon.fr from your instance, your instance queries https://oc.todon.fr/.well-known/webfinger?resource=acct:val@oc.todon.fr -18:57 < pinkieval> then it's supposed to fetch the account and show it, but it fails at that part -18:58 < pinkieval> sorry, I don't know where else to look -18:59 < pinkieval> I don't see anything obvious in the code related to that -18:59 < pinkieval> you should open an issue on github with all the details we mentioned here -19:00 < ptbridge> yeah OK, wow this has been super helpful. thanks much. -19:01 -!- maelsea [~maelsea@82.47.218.177] has quit [Quit: see ya] - [19:03] [ptbridge(+i)] [3:freenode/#mastodon(+cnt)] [Act: 4] -[#mastodon] -``` - - -searching for `paul@hptrow.me` from `https://social.linux.pizza/web/search` results in hptrow nginx log: -``` -54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /.well-known/webfinger?resource=acct:paul@hptrow.me HTTP/1.1" 200 425 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)" -54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /users/paul HTTP/1.1" 404 36 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)" -``` \ No newline at end of file +currently schema changes were not implemented and now the database is in an older state than the code. diff --git a/mastodon_issue.md b/mastodon_issue.md new file mode 100644 index 0000000..29186bf --- /dev/null +++ b/mastodon_issue.md @@ -0,0 +1,146 @@ +issue with mastodon-streaming service. + +``` +Jan 19 23:05:26 r710 node[17762]: /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:34 +Jan 19 23:05:26 r710 node[17762]: throw e.message = e.message + " check './node_modules/@clusterws/cws/build_log.txt' for post install build logs", +Jan 19 23:05:26 r710 node[17762]: ^ +Jan 19 23:05:26 r710 node[17762]: Error: The module '/home/mastodon/live/node_modules/@clusterws/cws/dist/cws_linux_79.node' +Jan 19 23:05:26 r710 node[17762]: was compiled against a different Node.js version using +Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 72. This version of Node.js requires +Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 79. Please try re-compiling or re-installing +Jan 19 23:05:26 r710 node[17762]: the module (for instance, using `npm rebuild` or `npm install`). check './node_modules/@clusterws/cws/build_log.txt' for post install build logs +Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..node (internal/modules/cjs/loader.js:1194:18) +Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32) +Jan 19 23:05:26 r710 node[17762]: at Function.Module._load (internal/modules/cjs/loader.js:892:14) +Jan 19 23:05:26 r710 node[17762]: at Module.require (internal/modules/cjs/loader.js:1033:19) +Jan 19 23:05:26 r710 node[17762]: at require (internal/modules/cjs/helpers.js:72:18) +Jan 19 23:05:26 r710 node[17762]: at /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:32:16 +Jan 19 23:05:26 r710 node[17762]: at Object. (/home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:37:3) +Jan 19 23:05:26 r710 node[17762]: at Module._compile (internal/modules/cjs/loader.js:1144:30) +Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:1164:10) +Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32) +``` +per [node webiste](https://nodejs.org/en/download/releases/) node module version 72 NodeJS v 12.14.1 and npm version 6.13.4 + +live/streaming hold the top level code. +if I try to run `node index.js` and hardcode the REDIS password, end up with postgres authentication error due to the connection module not supporting SCRAM-SHA-256 + +changed that back: real issue was that node was reverting to the latest version instead of LTS when starting the service since NVM is only per session. +apt n module makes a permanent version change, used that instead + +so trust fixes a manual run of index.js, but the having that available on port 4000 doesn't help the search function. + + +now it is clear that search issue doesn't have anything to do with resolved streaming API service. +shoudl try service maybe? + +found this in the issues: +https://github.com/tootsuite/mastodon/issues/5765 + + +This issue notes a web domain setting, grep of mastodon directory gives: +``` +./live/config/initializers/1_hosts.rb:web_host = ENV.fetch('WEB_DOMAIN') { host } +./live/lib/mastodon/premailer_webpack_strategy.rb: asset_host = ENV['CDN_HOST'] || ENV['WEB_DOMAIN'] || ENV['LOCAL_DOMAIN'] +./live/.env.nanobox:# WEB_DOMAIN=mastodon.example.com +./live/.env.nanobox:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN +./live/.env.nanobox:# if WEB_DOMAIN is not set. For example, the server may have the +./live/.env.nanobox:# The attachment host must allow cross origin request from WEB_DOMAIN or +./live/.env.nanobox:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the +./live/.env.production.sample:# WEB_DOMAIN=mastodon.example.com +./live/.env.production.sample:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN +./live/.env.production.sample:# if WEB_DOMAIN is not set. For example, the server may have the +./live/.env.production.sample:# The attachment host must allow cross origin request from WEB_DOMAIN or +./live/.env.production.sample:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the +``` + +notes in env.production.example say not to set `WEB_DOMAIN` + + + +posted a question on [discourse](https://discourse.joinmastodon.org/t/search-return-404/2490) + + +Issue with searching accounts on other instances + +syslog: +``` +Jan 30 20:35:50 r710 bundle[7118]: [a80db716-134c-4ca2-9c18-785675ec716d] method=GET path=/api/v2/search format=html controller=Api::V2::SearchController action=index status=404 duration=618.40 view=0.41 db=3.85 +``` + +nginx access.log +``` +[30/Jan/2020:20:35:50 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 54 "https://hptrow.me/" "Mozilla/5.0 (Android 8.1.0; Mobile; rv:68.0) Gecko/68.0 Firefox/68.0" +``` + +IRC convo: +``` +18:05 < pinkieval> what version is your instance? +18:13 < ptbridge> 3.0.1 +18:14 < pinkieval> mm +18:14 < ptbridge> yeah +18:15 < pinkieval> so apparently it recognized the URL (because of "controller=Api::V2::SearchController") but somehow still returns a 404 +18:15 < pinkieval> maybe ?q= is missing? +18:17 < ptbridge> nginx access.log -> "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" +18:17 < ptbridge> missed the 404 "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 +18:19 < pinkieval> https://mastodon.social/api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 is not a 404 +18:19 < pinkieval> what domain is your instance? +18:19 < ptbridge> hptrow.me +18:19 < pinkieval> I'm not getting a 404 either +18:20 < ptbridge> i just set it up and was messing around, and noticed search didn't seem to hook up outside of my instance +18:20 < pinkieval> I'm getting a 401 +18:21 < ptbridge> ¯\_(ツ)_/¯ +18:22 < ptbridge> thanks for checking +18:23 < pinkieval> how are you getting that 404? +18:24 < pinkieval> is it because you're using an API token? +18:26 < ptbridge> i haven't tried that, i just tried to do the search on the webpage and it didn't work so I checked my logs and that's what i saw +18:26 < pinkieval> uh +18:27 < ptbridge> like literally go up to the search box and type tusky@mastodon.social and nothing come back +18:27 < ptbridge> comes* +18:28 < pinkieval> I just created an account on your instance so I can test, can you approve it? +18:29 < pinkieval> I'm val +18:30 < ptbridge> done +18:31 < pinkieval> it's working for me +18:31 < ptbridge> wuh +18:31 < ptbridge> lol +18:31 < ptbridge> hang on let me check the logs +18:32 < ptbridge> sure enough -> "GET /api/v2/search?q=mastodon&resolve=true&limit=5 HTTP/2.0" 200 +18:33 < ptbridge> try searchign for an account on another instance +18:34 < pinkieval> oh right, 404 now +18:34 < pinkieval> that's weird +18:35 < ptbridge> yeah -> 51.159.34.167 - - [31/Jan/2020:18:34:12 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 +18:35 < ptbridge> oh well, thanks so much for checking that out! +18:36 < pinkieval> do you have Elasticsearch configured? +18:37 < ptbridge> yeah I have no idea, i saw mention of that, but don't see that as a service running +18:37 < pinkieval> so no +18:38 < pinkieval> so that's not the source of the issue +18:40 < pinkieval> ah! error response is: hptrow.me +18:40 < pinkieval> meh +18:40 < pinkieval> ah! error response is: {"error":"Record not found"} +18:41 < pinkieval> that's partially helpful +18:41 < ptbridge> does the API itself have a log it is dumping to somewhere? +18:42 < ptbridge> I just have a log for the result of the call to the API +18:43 < pinkieval> afaik, no +18:45 < ptbridge> It probably something dumb I didn't setup right with nginx or something +18:45 < ptbridge> alright, thanks so much +18:49 < pinkieval> your instance does reach mine when I make these requests, though: 174.104.140.112 "oc.todon.fr" - [31/Jan/2020:23:38:42 +0000] "GET /.well-known/webfinger?resource=acct:val@oc.todon.fr HTTP/1.1" 200 1649 "-" "http.rb/3.3.0 (Mastodon/3.0.1; + +https://hptrow.me/)" +18:52 -!- Pythayr [~pither195@38.132.120.242] has quit [Read error: Connection reset by peer] +18:54 < ptbridge> how is that request generated? +18:56 < pinkieval> when I search for @val@oc.todon.fr from your instance, your instance queries https://oc.todon.fr/.well-known/webfinger?resource=acct:val@oc.todon.fr +18:57 < pinkieval> then it's supposed to fetch the account and show it, but it fails at that part +18:58 < pinkieval> sorry, I don't know where else to look +18:59 < pinkieval> I don't see anything obvious in the code related to that +18:59 < pinkieval> you should open an issue on github with all the details we mentioned here +19:00 < ptbridge> yeah OK, wow this has been super helpful. thanks much. +19:01 -!- maelsea [~maelsea@82.47.218.177] has quit [Quit: see ya] + [19:03] [ptbridge(+i)] [3:freenode/#mastodon(+cnt)] [Act: 4] +[#mastodon] +``` + + +searching for `paul@hptrow.me` from `https://social.linux.pizza/web/search` results in hptrow nginx log: +``` +54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /.well-known/webfinger?resource=acct:paul@hptrow.me HTTP/1.1" 200 425 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)" +54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /users/paul HTTP/1.1" 404 36 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)" +``` \ No newline at end of file