pt
/
notes
1
0
Fork 0
Code Issues 1 Pull Requests Projects 1 Releases Wiki Activity

notes on mastodon situation

This commit is contained in:
Paul Trowbridge 2020-10-29 20:27:17 -04:00
parent 5a24555c5c
commit 212a945393
3 changed files with 154 additions and 149 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
git.md
View File

@ -3,18 +3,19 @@ Branches
============================================
### Adding Branches ###
### Adding Branches
* local: `git checkout -b <branch>`
* remote: `git push --set-upstream <remote> <branch>`
* track remote: `git checkout --track <origin>/<branch>`
### Deleting Branches ###
### Deleting Branches
* local: `git branch -d <name>`
* remote: `git push -d <remote> <name>`
* realize remote deletes: `git remote prune <remote>`
### Non-Standard Activities ###
### Non-Standard Activities
* merge only a single file into another branch `git checkout <branch> -- <file>`
* delete from repo and file system `git rm <file>`
@ -24,4 +25,4 @@ Config
=============================================
* set line ending behaviour `git config --global core.autocrlf true`
* store credentials `git config credential.helper store` or `git config credential.helper cache`
* store credentials `git config credential.helper store` or `git config credential.helper cache`

148
mastodon.md
View File

@ -1,146 +1,4 @@
issue with mastodon-streaming service.
Mastodon is ruby app.
you need a ruby environment manager just like node, need to be aware of this as ruby environment upgrades in combination with mastodon upgrades have not worked well.
```
Jan 19 23:05:26 r710 node[17762]: /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:34
Jan 19 23:05:26 r710 node[17762]: throw e.message = e.message + " check './node_modules/@clusterws/cws/build_log.txt' for post install build logs",
Jan 19 23:05:26 r710 node[17762]: ^
Jan 19 23:05:26 r710 node[17762]: Error: The module '/home/mastodon/live/node_modules/@clusterws/cws/dist/cws_linux_79.node'
Jan 19 23:05:26 r710 node[17762]: was compiled against a different Node.js version using
Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 72. This version of Node.js requires
Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 79. Please try re-compiling or re-installing
Jan 19 23:05:26 r710 node[17762]: the module (for instance, using `npm rebuild` or `npm install`). check './node_modules/@clusterws/cws/build_log.txt' for post install build logs
Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..node (internal/modules/cjs/loader.js:1194:18)
Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32)
Jan 19 23:05:26 r710 node[17762]: at Function.Module._load (internal/modules/cjs/loader.js:892:14)
Jan 19 23:05:26 r710 node[17762]: at Module.require (internal/modules/cjs/loader.js:1033:19)
Jan 19 23:05:26 r710 node[17762]: at require (internal/modules/cjs/helpers.js:72:18)
Jan 19 23:05:26 r710 node[17762]: at /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:32:16
Jan 19 23:05:26 r710 node[17762]: at Object.<anonymous> (/home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:37:3)
Jan 19 23:05:26 r710 node[17762]: at Module._compile (internal/modules/cjs/loader.js:1144:30)
Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:1164:10)
Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32)
```
per [node webiste](https://nodejs.org/en/download/releases/) node module version 72 NodeJS v 12.14.1 and npm version 6.13.4
live/streaming hold the top level code.
if I try to run `node index.js` and hardcode the REDIS password, end up with postgres authentication error due to the connection module not supporting SCRAM-SHA-256
changed that back: real issue was that node was reverting to the latest version instead of LTS when starting the service since NVM is only per session.
apt n module makes a permanent version change, used that instead
so trust fixes a manual run of index.js, but the having that available on port 4000 doesn't help the search function.
now it is clear that search issue doesn't have anything to do with resolved streaming API service.
shoudl try service maybe?
found this in the issues:
https://github.com/tootsuite/mastodon/issues/5765
This issue notes a web domain setting, grep of mastodon directory gives:
```
./live/config/initializers/1_hosts.rb:web_host = ENV.fetch('WEB_DOMAIN') { host }
./live/lib/mastodon/premailer_webpack_strategy.rb: asset_host = ENV['CDN_HOST'] || ENV['WEB_DOMAIN'] || ENV['LOCAL_DOMAIN']
./live/.env.nanobox:# WEB_DOMAIN=mastodon.example.com
./live/.env.nanobox:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN
./live/.env.nanobox:# if WEB_DOMAIN is not set. For example, the server may have the
./live/.env.nanobox:# The attachment host must allow cross origin request from WEB_DOMAIN or
./live/.env.nanobox:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the
./live/.env.production.sample:# WEB_DOMAIN=mastodon.example.com
./live/.env.production.sample:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN
./live/.env.production.sample:# if WEB_DOMAIN is not set. For example, the server may have the
./live/.env.production.sample:# The attachment host must allow cross origin request from WEB_DOMAIN or
./live/.env.production.sample:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the
```
notes in env.production.example say not to set `WEB_DOMAIN`
posted a question on [discourse](https://discourse.joinmastodon.org/t/search-return-404/2490)
Issue with searching accounts on other instances
syslog:
```
Jan 30 20:35:50 r710 bundle[7118]: [a80db716-134c-4ca2-9c18-785675ec716d] method=GET path=/api/v2/search format=html controller=Api::V2::SearchController action=index status=404 duration=618.40 view=0.41 db=3.85
```
nginx access.log
```
[30/Jan/2020:20:35:50 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 54 "https://hptrow.me/" "Mozilla/5.0 (Android 8.1.0; Mobile; rv:68.0) Gecko/68.0 Firefox/68.0"
```
IRC convo:
```
18:05 < pinkieval> what version is your instance?
18:13 < ptbridge> 3.0.1
18:14 < pinkieval> mm
18:14 < ptbridge> yeah
18:15 < pinkieval> so apparently it recognized the URL (because of "controller=Api::V2::SearchController") but somehow still returns a 404
18:15 < pinkieval> maybe ?q= is missing?
18:17 < ptbridge> nginx access.log -> "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0"
18:17 < ptbridge> missed the 404 "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404
18:19 < pinkieval> https://mastodon.social/api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 is not a 404
18:19 < pinkieval> what domain is your instance?
18:19 < ptbridge> hptrow.me
18:19 < pinkieval> I'm not getting a 404 either
18:20 < ptbridge> i just set it up and was messing around, and noticed search didn't seem to hook up outside of my instance
18:20 < pinkieval> I'm getting a 401
18:21 < ptbridge> ¯\_(ツ)_/¯
18:22 < ptbridge> thanks for checking
18:23 < pinkieval> how are you getting that 404?
18:24 < pinkieval> is it because you're using an API token?
18:26 < ptbridge> i haven't tried that, i just tried to do the search on the webpage and it didn't work so I checked my logs and that's what i saw
18:26 < pinkieval> uh
18:27 < ptbridge> like literally go up to the search box and type tusky@mastodon.social and nothing come back
18:27 < ptbridge> comes*
18:28 < pinkieval> I just created an account on your instance so I can test, can you approve it?
18:29 < pinkieval> I'm val
18:30 < ptbridge> done
18:31 < pinkieval> it's working for me
18:31 < ptbridge> wuh
18:31 < ptbridge> lol
18:31 < ptbridge> hang on let me check the logs
18:32 < ptbridge> sure enough -> "GET /api/v2/search?q=mastodon&resolve=true&limit=5 HTTP/2.0" 200
18:33 < ptbridge> try searchign for an account on another instance
18:34 < pinkieval> oh right, 404 now
18:34 < pinkieval> that's weird
18:35 < ptbridge> yeah -> 51.159.34.167 - - [31/Jan/2020:18:34:12 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404
18:35 < ptbridge> oh well, thanks so much for checking that out!
18:36 < pinkieval> do you have Elasticsearch configured?
18:37 < ptbridge> yeah I have no idea, i saw mention of that, but don't see that as a service running
18:37 < pinkieval> so no
18:38 < pinkieval> so that's not the source of the issue
18:40 < pinkieval> ah! error response is: hptrow.me
18:40 < pinkieval> meh
18:40 < pinkieval> ah! error response is: {"error":"Record not found"}
18:41 < pinkieval> that's partially helpful
18:41 < ptbridge> does the API itself have a log it is dumping to somewhere?
18:42 < ptbridge> I just have a log for the result of the call to the API
18:43 < pinkieval> afaik, no
18:45 < ptbridge> It probably something dumb I didn't setup right with nginx or something
18:45 < ptbridge> alright, thanks so much
18:49 < pinkieval> your instance does reach mine when I make these requests, though: 174.104.140.112 "oc.todon.fr" - [31/Jan/2020:23:38:42 +0000] "GET /.well-known/webfinger?resource=acct:val@oc.todon.fr HTTP/1.1" 200 1649 "-" "http.rb/3.3.0 (Mastodon/3.0.1;
+https://hptrow.me/)"
18:52 -!- Pythayr [~pither195@38.132.120.242] has quit [Read error: Connection reset by peer]
18:54 < ptbridge> how is that request generated?
18:56 < pinkieval> when I search for @val@oc.todon.fr from your instance, your instance queries https://oc.todon.fr/.well-known/webfinger?resource=acct:val@oc.todon.fr
18:57 < pinkieval> then it's supposed to fetch the account and show it, but it fails at that part
18:58 < pinkieval> sorry, I don't know where else to look
18:59 < pinkieval> I don't see anything obvious in the code related to that
18:59 < pinkieval> you should open an issue on github with all the details we mentioned here
19:00 < ptbridge> yeah OK, wow this has been super helpful. thanks much.
19:01 -!- maelsea [~maelsea@82.47.218.177] has quit [Quit: see ya]
[19:03] [ptbridge(+i)] [3:freenode/#mastodon(+cnt)] [Act: 4]
[#mastodon]
```
searching for `paul@hptrow.me` from `https://social.linux.pizza/web/search` results in hptrow nginx log:
```
54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /.well-known/webfinger?resource=acct:paul@hptrow.me HTTP/1.1" 200 425 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)"
54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /users/paul HTTP/1.1" 404 36 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)"
```
currently schema changes were not implemented and now the database is in an older state than the code.

146
mastodon_issue.md Normal file
View File

@ -0,0 +1,146 @@
issue with mastodon-streaming service.
```
Jan 19 23:05:26 r710 node[17762]: /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:34
Jan 19 23:05:26 r710 node[17762]: throw e.message = e.message + " check './node_modules/@clusterws/cws/build_log.txt' for post install build logs",
Jan 19 23:05:26 r710 node[17762]: ^
Jan 19 23:05:26 r710 node[17762]: Error: The module '/home/mastodon/live/node_modules/@clusterws/cws/dist/cws_linux_79.node'
Jan 19 23:05:26 r710 node[17762]: was compiled against a different Node.js version using
Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 72. This version of Node.js requires
Jan 19 23:05:26 r710 node[17762]: NODE_MODULE_VERSION 79. Please try re-compiling or re-installing
Jan 19 23:05:26 r710 node[17762]: the module (for instance, using `npm rebuild` or `npm install`). check './node_modules/@clusterws/cws/build_log.txt' for post install build logs
Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..node (internal/modules/cjs/loader.js:1194:18)
Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32)
Jan 19 23:05:26 r710 node[17762]: at Function.Module._load (internal/modules/cjs/loader.js:892:14)
Jan 19 23:05:26 r710 node[17762]: at Module.require (internal/modules/cjs/loader.js:1033:19)
Jan 19 23:05:26 r710 node[17762]: at require (internal/modules/cjs/helpers.js:72:18)
Jan 19 23:05:26 r710 node[17762]: at /home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:32:16
Jan 19 23:05:26 r710 node[17762]: at Object.<anonymous> (/home/mastodon/live/node_modules/@clusterws/cws/dist/index.js:37:3)
Jan 19 23:05:26 r710 node[17762]: at Module._compile (internal/modules/cjs/loader.js:1144:30)
Jan 19 23:05:26 r710 node[17762]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:1164:10)
Jan 19 23:05:26 r710 node[17762]: at Module.load (internal/modules/cjs/loader.js:993:32)
```
per [node webiste](https://nodejs.org/en/download/releases/) node module version 72 NodeJS v 12.14.1 and npm version 6.13.4
live/streaming hold the top level code.
if I try to run `node index.js` and hardcode the REDIS password, end up with postgres authentication error due to the connection module not supporting SCRAM-SHA-256
changed that back: real issue was that node was reverting to the latest version instead of LTS when starting the service since NVM is only per session.
apt n module makes a permanent version change, used that instead
so trust fixes a manual run of index.js, but the having that available on port 4000 doesn't help the search function.
now it is clear that search issue doesn't have anything to do with resolved streaming API service.
shoudl try service maybe?
found this in the issues:
https://github.com/tootsuite/mastodon/issues/5765
This issue notes a web domain setting, grep of mastodon directory gives:
```
./live/config/initializers/1_hosts.rb:web_host = ENV.fetch('WEB_DOMAIN') { host }
./live/lib/mastodon/premailer_webpack_strategy.rb: asset_host = ENV['CDN_HOST'] || ENV['WEB_DOMAIN'] || ENV['LOCAL_DOMAIN']
./live/.env.nanobox:# WEB_DOMAIN=mastodon.example.com
./live/.env.nanobox:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN
./live/.env.nanobox:# if WEB_DOMAIN is not set. For example, the server may have the
./live/.env.nanobox:# The attachment host must allow cross origin request from WEB_DOMAIN or
./live/.env.nanobox:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the
./live/.env.production.sample:# WEB_DOMAIN=mastodon.example.com
./live/.env.production.sample:# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN
./live/.env.production.sample:# if WEB_DOMAIN is not set. For example, the server may have the
./live/.env.production.sample:# The attachment host must allow cross origin request from WEB_DOMAIN or
./live/.env.production.sample:# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the
```
notes in env.production.example say not to set `WEB_DOMAIN`
posted a question on [discourse](https://discourse.joinmastodon.org/t/search-return-404/2490)
Issue with searching accounts on other instances
syslog:
```
Jan 30 20:35:50 r710 bundle[7118]: [a80db716-134c-4ca2-9c18-785675ec716d] method=GET path=/api/v2/search format=html controller=Api::V2::SearchController action=index status=404 duration=618.40 view=0.41 db=3.85
```
nginx access.log
```
[30/Jan/2020:20:35:50 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404 54 "https://hptrow.me/" "Mozilla/5.0 (Android 8.1.0; Mobile; rv:68.0) Gecko/68.0 Firefox/68.0"
```
IRC convo:
```
18:05 < pinkieval> what version is your instance?
18:13 < ptbridge> 3.0.1
18:14 < pinkieval> mm
18:14 < ptbridge> yeah
18:15 < pinkieval> so apparently it recognized the URL (because of "controller=Api::V2::SearchController") but somehow still returns a 404
18:15 < pinkieval> maybe ?q= is missing?
18:17 < ptbridge> nginx access.log -> "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0"
18:17 < ptbridge> missed the 404 "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404
18:19 < pinkieval> https://mastodon.social/api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 is not a 404
18:19 < pinkieval> what domain is your instance?
18:19 < ptbridge> hptrow.me
18:19 < pinkieval> I'm not getting a 404 either
18:20 < ptbridge> i just set it up and was messing around, and noticed search didn't seem to hook up outside of my instance
18:20 < pinkieval> I'm getting a 401
18:21 < ptbridge> ¯\_(ツ)_/¯
18:22 < ptbridge> thanks for checking
18:23 < pinkieval> how are you getting that 404?
18:24 < pinkieval> is it because you're using an API token?
18:26 < ptbridge> i haven't tried that, i just tried to do the search on the webpage and it didn't work so I checked my logs and that's what i saw
18:26 < pinkieval> uh
18:27 < ptbridge> like literally go up to the search box and type tusky@mastodon.social and nothing come back
18:27 < ptbridge> comes*
18:28 < pinkieval> I just created an account on your instance so I can test, can you approve it?
18:29 < pinkieval> I'm val
18:30 < ptbridge> done
18:31 < pinkieval> it's working for me
18:31 < ptbridge> wuh
18:31 < ptbridge> lol
18:31 < ptbridge> hang on let me check the logs
18:32 < ptbridge> sure enough -> "GET /api/v2/search?q=mastodon&resolve=true&limit=5 HTTP/2.0" 200
18:33 < ptbridge> try searchign for an account on another instance
18:34 < pinkieval> oh right, 404 now
18:34 < pinkieval> that's weird
18:35 < ptbridge> yeah -> 51.159.34.167 - - [31/Jan/2020:18:34:12 -0500] "GET /api/v2/search?q=tusky@mastodon.social&resolve=true&limit=5 HTTP/2.0" 404
18:35 < ptbridge> oh well, thanks so much for checking that out!
18:36 < pinkieval> do you have Elasticsearch configured?
18:37 < ptbridge> yeah I have no idea, i saw mention of that, but don't see that as a service running
18:37 < pinkieval> so no
18:38 < pinkieval> so that's not the source of the issue
18:40 < pinkieval> ah! error response is: hptrow.me
18:40 < pinkieval> meh
18:40 < pinkieval> ah! error response is: {"error":"Record not found"}
18:41 < pinkieval> that's partially helpful
18:41 < ptbridge> does the API itself have a log it is dumping to somewhere?
18:42 < ptbridge> I just have a log for the result of the call to the API
18:43 < pinkieval> afaik, no
18:45 < ptbridge> It probably something dumb I didn't setup right with nginx or something
18:45 < ptbridge> alright, thanks so much
18:49 < pinkieval> your instance does reach mine when I make these requests, though: 174.104.140.112 "oc.todon.fr" - [31/Jan/2020:23:38:42 +0000] "GET /.well-known/webfinger?resource=acct:val@oc.todon.fr HTTP/1.1" 200 1649 "-" "http.rb/3.3.0 (Mastodon/3.0.1;
+https://hptrow.me/)"
18:52 -!- Pythayr [~pither195@38.132.120.242] has quit [Read error: Connection reset by peer]
18:54 < ptbridge> how is that request generated?
18:56 < pinkieval> when I search for @val@oc.todon.fr from your instance, your instance queries https://oc.todon.fr/.well-known/webfinger?resource=acct:val@oc.todon.fr
18:57 < pinkieval> then it's supposed to fetch the account and show it, but it fails at that part
18:58 < pinkieval> sorry, I don't know where else to look
18:59 < pinkieval> I don't see anything obvious in the code related to that
18:59 < pinkieval> you should open an issue on github with all the details we mentioned here
19:00 < ptbridge> yeah OK, wow this has been super helpful. thanks much.
19:01 -!- maelsea [~maelsea@82.47.218.177] has quit [Quit: see ya]
[19:03] [ptbridge(+i)] [3:freenode/#mastodon(+cnt)] [Act: 4]
[#mastodon]
```
searching for `paul@hptrow.me` from `https://social.linux.pizza/web/search` results in hptrow nginx log:
```
54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /.well-known/webfinger?resource=acct:paul@hptrow.me HTTP/1.1" 200 425 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)"
54.37.254.26 - - [02/Feb/2020:19:59:55 -0500] "GET /users/paul HTTP/1.1" 404 36 "-" "http.rb/3.3.0 (Mastodon/3.0.1; +https://social.linux.pizza/)"
```