Feed Bot Issues

Forum changes and usage – feedback, requests, site updates
Post Reply
Skillz
Site Admin
Reactions:
Posts: 1315
Joined: Sun Sep 15, 2019 3:03 pm

Feed Bot Issues

Post by Skillz » Sat Jun 19, 2021 9:12 pm

So I have discovered an issue with the feed bot. Not sure how long this has been going on or if it's something new.

When the feed bot gets to a URL that is currently down then it will stop at that feed and not continue. This means, out of the 30+ feeds we are currently pulling it is stops at number 5 on the list, then 6 - 30 will not be checked.

I have reached out to the author of the extension to see if it's something on my end or if it's a problem with the extension.

I thought, the "timeout" column for each URL addresses that, but apparently not.

Added in 23 minutes 10 seconds:
Crap, I just noticed it has double posted A LOT of posts while I was trying to figure out the issue. Crap, crap, crap

StefanR5R
TAAT Member
Reactions:
Posts: 1168
Joined: Wed Sep 25, 2019 4:32 pm

Re: Feed Bot Issues

Post by StefanR5R » Sun Jun 20, 2021 4:31 am

Skillz wrote:
Sat Jun 19, 2021 9:35 pm
I thought, the "timeout" column for each URL addresses that, but apparently not.
I had been wondering about that too. Though if a simple timeout causes the bot to quit entirely, then we should apparently increase the timeout threshold in order to decrease the likelihood of timeouts. It is currently "3", which I presume is 3 seconds. How about making it "30" in each feed?

Skillz
Site Admin
Reactions:
Posts: 1315
Joined: Sun Sep 15, 2019 3:03 pm

Re: Feed Bot Issues

Post by Skillz » Sun Jun 20, 2021 9:10 am

I thought about that, too; however I think the issue is the fact that when the feed is down then it doesn't matter how long the time out is since it's going to time out anyway. Correct? I meant to keep track of both the feeds I had to remove so I could add them back later, but I found two feeds that when I went to the site directly their site/server was down. So no response from them at all.

StefanR5R
TAAT Member
Reactions:
Posts: 1168
Joined: Wed Sep 25, 2019 4:32 pm

Re: Feed Bot Issues

Post by StefanR5R » Sun Jun 20, 2021 10:53 am

Counter to intuition, it should be possible to disable a feed via its "feed forum" menu, top entry.

The "Feed processing has started but not completed and therefore cannot start again. If this persists you can release the process by clicking this button: Submit" condition still happens. I'll go ahead and increase the timeout to "10".

I am also disabling ODLK1's feed for now because it has got malformed XML. But since the bot was sometimes able to proceeed in the past, I suspect this particular issue does not throw off the bot.

Skillz
Site Admin
Reactions:
Posts: 1315
Joined: Sun Sep 15, 2019 3:03 pm

Re: Feed Bot Issues

Post by Skillz » Sun Jun 20, 2021 10:57 am

Ah that's how to disable it. I couldn't find it when I went looking to disable one that was offline.

I posted on ODLK1's site a little while ago about their feed being broken: https://boinc.multi-pool.info/latinsqua ... php?id=140

Maybe they'll fix it.

StefanR5R
TAAT Member
Reactions:
Posts: 1168
Joined: Wed Sep 25, 2019 4:32 pm

Re: Feed Bot Issues

Post by StefanR5R » Sun Jun 20, 2021 10:58 am

Beefathome times out, I am disabling it too.

StefanR5R
TAAT Member
Reactions:
Posts: 1168
Joined: Wed Sep 25, 2019 4:32 pm

Re: Feed Bot Issues

Post by StefanR5R » Wed Jul 07, 2021 6:13 pm

This game sux. I disabled Collatz now, as the site times out today and thus kills the feed bot. Need to remember to periodically check for when Collatz returns.

Skillz
Site Admin
Reactions:
Posts: 1315
Joined: Sun Sep 15, 2019 3:03 pm

Re: Feed Bot Issues

Post by Skillz » Wed Jul 07, 2021 6:31 pm

I need to edit the php settings to allow more than, I think, 30 seconds for it to timeout. I think since we have so many in there, by the time it gets to one that's broken/not working the 30 second timer is already almost done.

Give me a minute to get around to that here shortly.

I was also told that I need to enable something that'll help it, but I forget now. Have to go back to the phpbb forums, plugin help sub-forum to see what he told me I needed. I forgot about it.

Added in 46 minutes 24 seconds:
The extension he mentioned was curl, curl is installed on the server, but I can't get PHP to build with CURL. So I opened a support ticket to see if I can figure that out.

StefanR5R
TAAT Member
Reactions:
Posts: 1168
Joined: Wed Sep 25, 2019 4:32 pm

Re: Feed Bot Issues

Post by StefanR5R » Thu Jul 08, 2021 2:48 pm

Back up: Collatz, TBrada. :-)
Newly down: iThena, Gaia. :-(

Skillz
Site Admin
Reactions:
Posts: 1315
Joined: Sun Sep 15, 2019 3:03 pm

Re: Feed Bot Issues

Post by Skillz » Thu Jul 08, 2021 2:57 pm

This is becoming a full time job for you and that wasn't my intentions. I need to figure out a better way to accomplish this so we don't have to babysit it all the time.

StefanR5R
TAAT Member
Reactions:
Posts: 1168
Joined: Wed Sep 25, 2019 4:32 pm

Re: Feed Bot Issues

Post by StefanR5R » Thu Jul 08, 2021 3:06 pm

Don't worry; I already tend to a different full-time job, one which actually pays. :-)

StefanR5R
TAAT Member
Reactions:
Posts: 1168
Joined: Wed Sep 25, 2019 4:32 pm

Re: Feed Bot Issues

Post by StefanR5R » Tue Jul 13, 2021 2:35 pm

Is it normal that the bot pulls the feeds much more frequently than configured at "Interval for automatic processing feeds (seconds)"? The Maintenance/ Admin log shows that it happens much more often, and at varying periods.

Skillz
Site Admin
Reactions:
Posts: 1315
Joined: Sun Sep 15, 2019 3:03 pm

Re: Feed Bot Issues

Post by Skillz » Tue Jul 13, 2021 2:59 pm

I don't think it should.

Added in 3 minutes 37 seconds:
Seems when a user visits the site it triggers the bot. Not sure how to disable that. Will look into it when I get home this afternoon.

Post Reply