Page 1 of 1

Feed Bot Issues

Posted: Sat Jun 19, 2021 9:35 pm
by Skillz
So I have discovered an issue with the feed bot. Not sure how long this has been going on or if it's something new.

When the feed bot gets to a URL that is currently down then it will stop at that feed and not continue. This means, out of the 30+ feeds we are currently pulling it is stops at number 5 on the list, then 6 - 30 will not be checked.

I have reached out to the author of the extension to see if it's something on my end or if it's a problem with the extension.

I thought, the "timeout" column for each URL addresses that, but apparently not.

Added in 23 minutes 10 seconds:
Crap, I just noticed it has double posted A LOT of posts while I was trying to figure out the issue. Crap, crap, crap

Re: Feed Bot Issues

Posted: Sun Jun 20, 2021 4:31 am
by StefanR5R
Skillz wrote: Sat Jun 19, 2021 9:35 pm I thought, the "timeout" column for each URL addresses that, but apparently not.
I had been wondering about that too. Though if a simple timeout causes the bot to quit entirely, then we should apparently increase the timeout threshold in order to decrease the likelihood of timeouts. It is currently "3", which I presume is 3 seconds. How about making it "30" in each feed?

Re: Feed Bot Issues

Posted: Sun Jun 20, 2021 9:10 am
by Skillz
I thought about that, too; however I think the issue is the fact that when the feed is down then it doesn't matter how long the time out is since it's going to time out anyway. Correct? I meant to keep track of both the feeds I had to remove so I could add them back later, but I found two feeds that when I went to the site directly their site/server was down. So no response from them at all.

Re: Feed Bot Issues

Posted: Sun Jun 20, 2021 10:53 am
by StefanR5R
Counter to intuition, it should be possible to disable a feed via its "feed forum" menu, top entry.

The "Feed processing has started but not completed and therefore cannot start again. If this persists you can release the process by clicking this button: Submit" condition still happens. I'll go ahead and increase the timeout to "10".

I am also disabling ODLK1's feed for now because it has got malformed XML. But since the bot was sometimes able to proceeed in the past, I suspect this particular issue does not throw off the bot.

Re: Feed Bot Issues

Posted: Sun Jun 20, 2021 10:57 am
by Skillz
Ah that's how to disable it. I couldn't find it when I went looking to disable one that was offline.

I posted on ODLK1's site a little while ago about their feed being broken: https://boinc.multi-pool.info/latinsqua ... php?id=140

Maybe they'll fix it.

Re: Feed Bot Issues

Posted: Sun Jun 20, 2021 10:58 am
by StefanR5R
Beefathome times out, I am disabling it too.

Re: Feed Bot Issues

Posted: Wed Jul 07, 2021 6:13 pm
by StefanR5R
This game sux. I disabled Collatz now, as the site times out today and thus kills the feed bot. Need to remember to periodically check for when Collatz returns.

Re: Feed Bot Issues

Posted: Wed Jul 07, 2021 7:18 pm
by Skillz
I need to edit the php settings to allow more than, I think, 30 seconds for it to timeout. I think since we have so many in there, by the time it gets to one that's broken/not working the 30 second timer is already almost done.

Give me a minute to get around to that here shortly.

I was also told that I need to enable something that'll help it, but I forget now. Have to go back to the phpbb forums, plugin help sub-forum to see what he told me I needed. I forgot about it.

Added in 46 minutes 24 seconds:
The extension he mentioned was curl, curl is installed on the server, but I can't get PHP to build with CURL. So I opened a support ticket to see if I can figure that out.

Re: Feed Bot Issues

Posted: Thu Jul 08, 2021 2:48 pm
by StefanR5R
Back up: Collatz, TBrada. :-)
Newly down: iThena, Gaia. :-(

Re: Feed Bot Issues

Posted: Thu Jul 08, 2021 2:57 pm
by Skillz
This is becoming a full time job for you and that wasn't my intentions. I need to figure out a better way to accomplish this so we don't have to babysit it all the time.

Re: Feed Bot Issues

Posted: Thu Jul 08, 2021 3:06 pm
by StefanR5R
Don't worry; I already tend to a different full-time job, one which actually pays. :-)

Re: Feed Bot Issues

Posted: Tue Jul 13, 2021 2:35 pm
by StefanR5R
Is it normal that the bot pulls the feeds much more frequently than configured at "Interval for automatic processing feeds (seconds)"? The Maintenance/ Admin log shows that it happens much more often, and at varying periods.

Re: Feed Bot Issues

Posted: Tue Jul 13, 2021 3:02 pm
by Skillz
I don't think it should.

Added in 3 minutes 37 seconds:
Seems when a user visits the site it triggers the bot. Not sure how to disable that. Will look into it when I get home this afternoon.

Re: Feed Bot Issues

Posted: Fri May 13, 2022 10:36 am
by StefanR5R
Every now and then the feed bot is blocked by iThena. (Most of the time iThena does respond, but eventually it doesn't and then the bot needs manual intervention to get going again.) So I would rather like to leave the two iThena newsfeeds disabled for now.

Re: Feed Bot Issues

Posted: Fri May 13, 2022 12:47 pm
by Skillz
I'm okay with that.

Re: Feed Bot Issues

Posted: Sat Oct 28, 2023 5:02 am
by StefanR5R
There is a new issue, perhaps because of the site migration:

"Feed processing has started but not completed and therefore cannot start again. If this persists you can release the process by clicking this button: [Submit]" --> Clicking [Submit] clears this state, as before. But:

"Fetch all feeds manually: [Submit]" --> Clicking [Submit] moves you to a blank page and does not start the bot. Or at least, the maintenance log does not show any bot activity.

Re: Feed Bot Issues

Posted: Mon Nov 06, 2023 2:06 am
by Skillz
I will be looking for an updated version or a new feedbot for the latest version of the forum.

edit
Their might be a way to select how old of a post it searches for in the feeds and then posts any from that point forward. If we set a cronjob on the server that definitely wont ever fail w/ those lines commented out then lower how many days/hours it checks back we might be able to prevent that somehow.

Re: Feed Bot Issues

Posted: Sun Nov 19, 2023 5:08 am
by StefanR5R
Since Sat Nov 18 at the latest, circa Wed Nov 15 at the earliest, "Fetch all feeds manually" takes no time but immediately returns with "No (new) items to fetch", which seems suspicious.

Re: Feed Bot Issues

Posted: Sun Nov 19, 2023 12:35 pm
by Skillz
Same result for me as well.

I looked around a few of the BOINC projects and didn't see any new posts. I'll check more later to see if any of them do have posts since then.

Re: Feed Bot Issues

Posted: Wed Nov 22, 2023 6:29 pm
by Skillz
Okay, its still doing the same thing and I know for sure at least one project (RakeSearch) has posted news recently. Will be looking into this tomorrow.