Aurora 4x

Off Topic => Forum Issues => Topic started by: MarcAFK on October 04, 2015, 08:14:29 AM

Title: 403 Forbidden
Post by: MarcAFK on October 04, 2015, 08:14:29 AM
I'm getting the following while trying to post in this thread http://aurora2.pentarch.org/index.php?topic=7956.0
" Forbidden

You don't have permission to access /index.php on this server. "

Edit: it's worse than that, I managed to post the word test, then later edit the post , but it's giving me the error when I post a paragraph.
I think we have forbidden words again :(
Title: Re: 403 Forbidden
Post by: MarcAFK on October 04, 2015, 08:33:31 AM
I managed to split it into 2, but pasting them together breaks it.
Title: Re: 403 Forbidden
Post by: Vandermeer on October 22, 2015, 04:26:55 AM
Same problem just appeared for me in my own sub-forum. Had to split a reply to my own post into 3 for it to work without getting above mentioned error.
I first thought it was simply length limitation, but my first post there is way longer then the combination of 2 of the other 3 at least, - if not all 3 combined. Not sure what happened.
Title: Re: 403 Forbidden
Post by: MarcAFK on October 22, 2015, 06:49:09 AM
Through a process of elimination I determined the exact statement which screwed up my post.
Any post containing case if then in that order will cause the error if all words have a space before and after them, no matter how many intervening words there are.
Also your old friend like is giving me grief too.
Putting the offending words into italics fixes it as always.
Edit: I couldn't post your last 3 posts  combined , until I put case from the middle post into italics, then it worked.
Title: Re: 403 Forbidden
Post by: Vandermeer on October 22, 2015, 07:39:00 AM
Ohh, thanks for the analysis. So the old problem is back? I bet it came back with the Star Wars sequels.
Seems weird though that the middle post, who had the case, was still deployable when posted individually. Before I could not post any word non-grata part no matter how much I separated it. I remember having to delete them and insert [blank] or something, before the revolutionary italic technology had been invented.
Title: Re: 403 Forbidden
Post by: Vandermeer on October 28, 2015, 06:49:00 AM
I have another problem that also yields '403 you are not allowed to access index.php' type error, when I try to get a preview on a post.
I tested around, and the malicious word seems to be "from". Just in the first sentence of the post it says: "[...] better accomplishments, aside from (quite literally) [...]".
If I cut the from out or make it italic, everything is ok. Otherwise, no preview, and no posting possible.

This is also true for this very post I am just posting, but strangely, just one week ago I could forge previews without issue, and I doubt I haven't used the word "from" somewhere in these huge text blocks. What a moody technology.
Title: Re: 403 Forbidden
Post by: Erik L on October 28, 2015, 08:33:06 AM
I have another problem that also yields '403 you are not allowed to access index.php' type error, when I try to get a preview on a post.
I tested around, and the malicious word seems to be "from". Just in the first sentence of the post it says: "[...] better accomplishments, aside from (quite literally) [...]".
If I cut the from out or make it italic, everything is ok. Otherwise, no preview, and no posting possible.

This is also true for this very post I am just posting, but strangely, just one week ago I could forge previews without issue, and I doubt I haven't used the word "from" somewhere in these huge text blocks. What a moody technology.

They are all SQL keywords. There is a fix, I just need to find time at home to implement it.
Title: Re: 403 Forbidden
Post by: Steve Walmsley on November 02, 2015, 06:02:28 PM
I just ran into a similar problem. I've been trying for a while to post an update in my Colonial Wars campaign and finally narrowed it down to the word "case" in the middle of a paragraph.

Surrounding words were "jump point in case the Brazilians"

As soon as I deleted "case" from the sentence, I could post.
Title: Re: 403 Forbidden
Post by: Vandermeer on November 05, 2015, 12:22:01 AM
Even older topics get conflicted. I just tried to update the Carrier Operations Tutorial from before, and while not adding much still got the error window, because the old text contained some words non-grata. (was another luckily lonely case btw.)
//That should be proof that the problem is indeed new again.
Title: Re: 403 Forbidden
Post by: 83athom on November 06, 2015, 09:12:31 AM
Lets see if this works. I am trying to post something in the Chat, can't seem to be able to cause 403.
Test one; Failure.
Test two; putting it in code; Failure
Test three; putting the known forbidden words in italics; Failure
Test four; putting whole post in italics; Failure
Test five; removing sentences; PASS. Turned out it was my first sentence that was giving me problems.
Well I found where
some npr is having fun
Its putting a . after fun that is making the sentence forbidden.
Title: Re: 403 Forbidden
Post by: Erik L on November 06, 2015, 09:26:03 AM
I know the fix for this. It is basically disabling script security for the site. I'm debating on if that is good or no.
Title: Re: 403 Forbidden
Post by: MarcAFK on November 06, 2015, 04:14:08 PM
Log a bug with whoever designed the script security code?
Then wait for it to be fixed, or alternatively a hacky solution would be create some code which first intercepts known error creating statements and shoves them into italics. Then leave script security alone.
Title: Re: 403 Forbidden
Post by: se5a on November 14, 2015, 05:11:31 PM
I'm getting this error attempting to view the wiki

http://aurorawiki.pentarch.org/
Title: Re: 403 Forbidden
Post by: Erik L on November 14, 2015, 08:24:11 PM
I'm getting this error attempting to view the wiki

http://aurorawiki.pentarch.org/

The wiki is currently offline.
Title: Re: 403 Forbidden
Post by: se5a on November 22, 2015, 09:12:11 PM
Ah right.  The little news blurb at the top right of the forum page said otherwise...
Title: Re: 403 Forbidden
Post by: Zokk on November 25, 2015, 01:08:13 AM
Quote from: Erik Luken link=topic=7958. msg82701#msg82701 date=1447554251
The wiki is currently offline.

Is there an ETA for when it'll be back up and running? I just started learning Aurora a few days ago, and I'm still mostly just fumbling around with it, even after looking up various tutorials online.
Title: Re: 403 Forbidden
Post by: Erik L on November 25, 2015, 08:19:28 AM
The wiki has been consuming a lot of resources on the host server. This leads them to throttle the entire domain.
Title: Re: 403 Forbidden
Post by: 83athom on November 25, 2015, 11:26:31 AM
Is there an ETA for when it'll be back up and running? I just started learning Aurora a few days ago, and I'm still mostly just fumbling around with it, even after looking up various tutorials online.
All the information on the wiki can be found here, you just have to look for it.
Title: Re: 403 Forbidden
Post by: Thundercraft on November 28, 2015, 09:36:00 AM
Wow! Am I glad I found this topic! I hadn't posted since 2013 and I was wondering if my 403 errors had something to do with my account. I just couldn't seem to start a new topic, no matter how many times I tried.

Now I just have to remember to always search my posts for from, like, and case and always put italics brackets around them.

I know the fix for this. It is basically disabling script security for the site. I'm debating on if that is good or no.
Log a bug with whoever designed the script security code?

Perhaps you could disable script security for the time being, report the bug to the forum software, and see how it goes until they have a fix?

...alternatively a hacky solution would be create some code which first intercepts known error creating statements and shoves them into italics. Then leave script security alone.

Sounds like it might work.

The wiki has been consuming a lot of resources on the host server. This leads them to throttle the entire domain.

Why? Was the wiki getting that much legitimate traffic? Were bots targeting it? Or, perhaps there's other wiki software that's not as resource intensive?

However, these days, spammers and spam-bots very frequently target wikis to insert their advertising junk. I know because I used to participate on the Dwarf Fortress wiki (and others) and some of the moderators there talked about it. I also saw it firsthand, with spam and junk added daily and new accounts created faster than they could delete them. Sometimes, wikis and forums have to disable account creation just to manage.

It's not enough, these days, for a wiki to just have a run-of-the-mill CAPTCHA. You need really good bot filtering, preferably in combo with a Q & A and account verification.

It's sad that the wiki had to be taken down. The biggest obstacle to getting into Aurora is the complexity. And a wiki would sure help with that. I wouldn't have even attempted to get into Dwarf Fortress if they didn't have a wiki.

Perhaps the wiki could be moved to a different location? Perhaps a free wiki host? Or maybe try scrapping the wiki membership and have much more strict bot checks and controls on registration? (Maybe ask prospective members why they want to join? Ask a question about Aurora?)
Title: Re: 403 Forbidden
Post by: Erik L on November 28, 2015, 12:21:24 PM
Why? Was the wiki getting that much legitimate traffic? Were bots targeting it? Or, perhaps there's other wiki software that's not as resource intensive?

However, these days, spammers and spam-bots very frequently target wikis to insert their advertising junk. I know because I used to participate on the Dwarf Fortress wiki (and others) and some of the moderators there talked about it. I also saw it firsthand, with spam and junk added daily and new accounts created faster than they could delete them. Sometimes, wikis and forums have to disable account creation just to manage.

It's not enough, these days, for a wiki to just have a run-of-the-mill CAPTCHA. You need really good bot filtering, preferably in combo with a Q & A and account verification.

It's sad that the wiki had to be taken down. The biggest obstacle to getting into Aurora is the complexity. And a wiki would sure help with that. I wouldn't have even attempted to get into Dwarf Fortress if they didn't have a wiki.

Perhaps the wiki could be moved to a different location? Perhaps a free wiki host? Or maybe try scrapping the wiki membership and have much more strict bot checks and controls on registration? (Maybe ask prospective members why they want to join? Ask a question about Aurora?)

The wiki has been on moderator account creation for a few years now. And only people with accounts can make changes. So it wasn't traffic to my knowledge.
Title: Re: 403 Forbidden
Post by: MarcAFK on December 21, 2015, 02:13:53 PM
I've found another thing that triggers the 403, any posts containing the following;
[any word] is [decimal number].
Example ;
"Euphrosyne is 0.0679"
Title: Re: 403 Forbidden
Post by: Erik L on December 21, 2015, 03:23:15 PM
Test is 1.234
Title: Re: 403 Forbidden
Post by: Erik L on December 21, 2015, 03:33:27 PM
Hmm. I implemented the suggested fix on the SMF forums, but that didn't seem to work
Title: Re: 403 Forbidden
Post by: MarcAFK on December 21, 2015, 04:23:27 PM
Oddly enough it has to be under 1.
Word is 0.99 will fail.
Title: Re: 403 Forbidden
Post by: Mor on December 22, 2015, 07:16:53 AM
Why? Was the wiki getting that much legitimate traffic?

This site setting bars it from google grabby little crawlers, meaning that any person that googles anything about the game will never get any search results leading here, only the wiki. So in effect the wiki is the only source of info as far as the world google is concerned.

All the information on the wiki can be found here, you just have to look for it.

Easier said than done. Even if you know how to get the most of the abhorrent forum search function, you are still handicapped.
Title: Re: 403 Forbidden
Post by: MarcAFK on December 22, 2015, 07:38:44 AM
What specifically is wrong with the search function? I've had issues with finding things before but usually manage to find what I need, I can't think of anything specifically wrong with it though.
Title: Re: 403 Forbidden
Post by: Erik L on December 22, 2015, 08:38:37 AM
The issue seems to be mod_security. I attempted to disable it for the Aurora forums, but that did not seem to work. At least some of the errors are still occurring.
Title: Re: 403 Forbidden
Post by: Mor on December 24, 2015, 03:58:57 AM
On the wiki, I no longer get this error, instead i get redirected to the main page.
Also, the site\wiki blasphemous term for today is "Description".
Title: Re: 403 Forbidden
Post by: Steve Walmsley on December 24, 2015, 01:52:24 PM
Just had this problem. Removed 0.49 from end of sentence (after narrowing it down) from whole update post :) and it started working.

After reading above, I changed the preceding text so it would read "reduced to 0.49", instead of "is 0.49" and it worked fine.
Title: Re: 403 Forbidden
Post by: se5a on December 27, 2015, 05:36:49 PM
What specifically is wrong with the search function? I've had issues with finding things before but usually manage to find what I need, I can't think of anything specifically wrong with it though.
Its  a pita to use,  the signal to noise ratio is poor unless you select the right subforums.  Wiki is far far easier to use.  Esp for new players.  Why is the wiki using so much data?
Title: Re: 403 Forbidden
Post by: Erik L on December 27, 2015, 08:34:06 PM
Why is the wiki using so much data?

I wish I knew why it was.
Title: Re: 403 Forbidden
Post by: Mor on December 31, 2015, 02:53:09 PM
Wiki is far far easier to use.  Esp for new players.  Why is the wiki using so much data?

Like I previously said, this forum setting bars it from google grabby little crawlers. Which means that as far as the world concerned all the info about the game (tutorials\change logs\etc) found only on the wiki, and this is where everyone are directed at. So the wiki is using soo much data because Aurora is soo popular and the wiki is the only site on the tour guide...

I assume that the sever\bandwidth can't be upgraded, but this might be addressed though the use of cache servers. Depending on how much traffic comes this way, there might be free solutions for that.
Title: Re: 403 Forbidden
Post by: Erik L on December 31, 2015, 04:31:07 PM
Like I previously said, this forum setting bars it from google grabby little crawlers. Which means that as far as the world concerned all the info about the game (tutorials\change logs\etc) found only on the wiki, and this is where everyone are directed at. So the wiki is using soo much data because Aurora is soo popular and the wiki is the only site on the tour guide...

I assume that the sever\bandwidth can't be upgraded, but this might be addressed though the use of cache servers. Depending on how much traffic comes this way, there might be free solutions for that.

I've updated that, along with adding a sitemap. So in theory, things should appear on google, etc.
Title: Re: 403 Forbidden
Post by: Mor on January 01, 2016, 01:54:52 AM
Great news for me, since I prefer using google search in most cases, but overall I doubt it will do much to address the issue at hand. Since the forum\wiki are hosted on the same sever it maters not how you balance he access traffic, it will choke anyway.
Title: Re: 403 Forbidden
Post by: Erik L on January 01, 2016, 11:40:14 AM
Great news for me, since I prefer using google search in most cases, but overall I doubt it will do much to address the issue at hand. Since the forum\wiki are hosted on the same sever it maters not how you balance he access traffic, it will choke anyway.

I know :(
Title: Re: 403 Forbidden
Post by: boggo2300 on January 03, 2016, 02:46:09 PM
My first return visit here since mid December, and I'm getting 500 internal server errors on just about every thread I open,  hit back, give it a second and try again and it usually works

errors I'm getting

Code: [Select]
Fatal error: Out of memory (allocated 786432) (tried to allocate 196608 bytes) in /home/penta6/public_html/aurora2/Sources/Subs.php on line 1760
Code: [Select]
Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at webmaster@aurora2.pentarch.org to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.
Title: Re: 403 Forbidden
Post by: demonblackie on January 03, 2016, 03:24:49 PM
Just here to second what boggo2300 said.   Every time I load a page in either the forum or the wiki, I'm getting the Internal Server Error.   Refresh the page, it loads.   It's been doing it for the past couple of days.
Title: Re: 403 Forbidden
Post by: Mor on January 07, 2016, 05:19:30 AM
^ this is due to peek hours usage. During the holidays Aurora got a lot more exposure than what the server can handle.

While the issue mentioned by the OP, is most likely an configuration issue or an evil plan by the host.
Title: Re: 403 Forbidden
Post by: Vandermeer on January 07, 2016, 10:12:59 AM
A new occasionally forbidden word joins the club. "ha ving" (no space)
I was stumbled when I couldn't find any of the usual trigger words in one recent and short post. After some back and forth, the error vanished when deleting the word, though I am sure I have used it other place often enough without causing any 403.
Well, in this case not even the cursive method helped. I had to change it into "to have" to make the post work.

On other accounts, without this incident, I didn't see much 403s for a long time.
Title: Re: 403 Forbidden
Post by: Erik L on January 07, 2016, 11:50:02 AM
The 403s seem to have been usurped by the out of memory and 500 errors.
Title: Re: 403 Forbidden
Post by: MarcAFK on January 07, 2016, 04:54:32 PM
The 403s seem to have been usurped by the out of memory and 500 errors.
Perhaps we need to download more ram?
Title: Re: 403 Forbidden
Post by: Mor on January 07, 2016, 06:09:13 PM
I've updated that, along with adding a sitemap. So in theory, things should appear on google, etc.
Indeed it is, and I can now use google (and its cache) to find things on the forum without having to stumble into host of errors  ::) However, I am bufleld  by the fact that your change caused this to be no longer the case for the wiki.

...
You got lucky. I have had the pleasure quit often this week, ironically most recently it was while trying to quote your post...  Overall, it makes no sense, its like the host is playing a hat tick, pulling random words... how can he not notice the issue and fix it?! that or he is an evil genius  ;)
Title: Re: 403 Forbidden
Post by: Erik L on January 07, 2016, 06:22:36 PM
Indeed it is, and I can now use google (and its cache) to find things on the forum without having to stumble into host of errors  ::) However, I am bufleld  by the fact that your change caused this to be no longer the case for the wiki.
You got lucky. I have had the pleasure quit often this week, ironically most recently it was while trying to quote your post...  Overall, it makes no sense, its like the host is playing a hat tick, pulling random words... how can he not notice the issue and fix it?! that or he is an evil genius  ;)

All of the words that have been triggers have been SQL keywords.

On a side note, the host suggests that I optimize images and consolidate CSS files. What. The. frakk.

On a second side note, I am now shopping for a domain host after 10+ years.

Bit of trivia... There has been ~450g of bandwidth HTTP transfer. Since Jan 1. If this keeps up, I'm looking at 2-2.5tb transfer/month.
Title: Re: 403 Forbidden
Post by: Erik L on January 07, 2016, 06:26:19 PM
This is what I am looking for from a host.

Domain requirements
•   10+ gig storage
•   2tb+ bandwidth
•   MySQL
•   Php 5.5+
•   Subdomain support
•   Domain/subdomain forwarding
•   Unlimited email accounts
•   POP3 and webmail support
•   FTP/SFTP support
•   Shell access

This should cover everything I have at the moment.
Title: Re: 403 Forbidden
Post by: MarcAFK on January 07, 2016, 11:27:03 PM
.... I know the forum activity exploded but that still seems excessive considering this is still a fairly small forum.
But images are a problem, even if spoilered they're always loaded. Perhaps SMF has a setting that might help somewhat, maybe storing and loading a small thumbnail unless clicked on ?
Title: Re: 403 Forbidden
Post by: Erik L on January 07, 2016, 11:36:59 PM
That'd be great, but all of the built-in image compression in SMF relies on software the host has not loaded.

December's traffic was ~150g.
January is 3x that a week in.
Title: Re: 403 Forbidden
Post by: MarcAFK on January 08, 2016, 12:04:12 AM
That'd be great, but all of the built-in image compression in SMF relies on software the host has not loaded.

December's traffic was ~150g.
January is 3x that a week in.
Oh hi, please optimise your images using software we won't load onto the server, kthxbye.
Edit: perhaps my maths are off, but there's 340,000 page views. That's an Average of 1.2 mb per page views.
Oh I'm an idiot, the game is downloaded from the forum right?
The 7.1 patch thread was viewed 12000 times since December, assuming it's been downloaded that many times and the 5.54 as well, would be 1.2 Tb.....
Surely it's a little lower than that but still, I'm sure you can check many times the games been downloaded....
If it's significant then hosting the files elsewhere should help a little.
Title: Re: 403 Forbidden
Post by: Erik L on January 08, 2016, 12:29:24 AM
Yeah, they are certainly annoying me to no end.
Title: Re: 403 Forbidden
Post by: Mor on January 08, 2016, 03:13:16 AM
On a side note, the host suggests that I optimize images and consolidate CSS files.
He is not completely wrong, it might alleviate some strain, every little bit helps, no? But here is catch 22, I tried to trim down the css on the wiki (I can reduce about a third without too much hassle) but I can't update it because 403 Forbidden!  At least images aren't an issue there, because of volume and the way mediawiki handles it.

.... I know the forum activity exploded but that still seems excessive considering this is still a fairly small forum.
Unfortunately, once you hit the ceiling, it usually tuns into a downward spiral. That is when people get an error they tend to obsessively refresh, generating extra calls for the already resource deprived sever..

The 7.1 patch thread was viewed 12000 times since December, assuming it's been downloaded that many times and the 5.54 as well, would be 1.2 Tb.....
Yeah, its most likely the single most offender as far as Eriks bandwidth stats goes. Probably more so for the portable version, but at least its not hosted here. Also it appears that most of the new users seem to have found a home on reddit.

Looks like Erik, changed the Aurora files hosting location, which should help. Btw, big thanks to Erik for working on all of this in the background!

Edited.
Title: Re: 403 Forbidden
Post by: Mor on January 09, 2016, 03:11:08 AM
All of the words that have been triggers have been SQL keywords.

All I can tell is that if was something consistent that I can identify, then I could write couple of regex fixes for myself (like i did for 0.x). but to me it seem too random. Like MarcAFK post above which several minutes later you can no longer quote, or from today:

On Version_7.0 (http://aurorawiki.pentarch.org/index.php?title=Version_7.0) I can't add:
Code: [Select]
==Army Officer Promotions==
only
;=Army Officer Promotions==
while there are plenty of other headers above and bellow just like that, but only this one fires, even if I remove the content

Or on Version_6.40 (http://aurorawiki.pentarch.org/index.php?title=Version_6.40), there are several trigger words including wh_en that prevent me from updating it.

Edit:
Whatever it is, it interferes with forum\wiki basic function, affecting people ability to post content and wasting their time trying to hunt the heretic word of the day.
Title: Re: 403 Forbidden
Post by: ardem on January 10, 2016, 05:58:01 PM
Lots of issues over the last two days for the forum for me. regardless where I am and what computers I use. I got a Server is out of resources message only 30 minutes ago. I had cannot post message forbidden. there was nothing in the text that I could see that was like 'case', but it would not allow me to post two sentences. I had very long delays in opening pages, sometimes not opening at all.

I think think the DB may need some maintenance.

Title: Re: 403 Forbidden
Post by: Ostia on January 11, 2016, 04:22:48 AM
We already discussed that here (http://aurora2.pentarch.org/index.php?topic=7798.0)

TL,DR: The hoster installed crap and now the server is in a constant state of near breakdown. Erik is on it and plans on switching hosts.