I Have Changed the Way Attachments Work

[GAD]

For the better!

First my problem: everyone LOVES attachments and since I enabled the feature in 2000 y'all have uploaded 48,818 images for a total of 20.67 GiB of disk space consumed. Woof!

While everyone likes to think that "disk is cheap" that's not really the case when it comes to hosted virtual machines, and those images are consuming 15% of the available disk space on the server which makes LTG attachments the single largest consumer of disk space on my system which includes 22 websites! At the rate it's being consumed the disk will be full in only a year or two, and that's not great.

Now your problem: Have you ever tried to upload an image from your phone or computer only to have the system tell you that it's too large? That's driven me *crazy* since I enabled the feature because I couldn't figure out why it was happeneing.

What I've changed: First, I discovered that the largest image that could be uploaded due to a default (and thus not written) limit of the software was 20 Megapixels. Given the current state of mobile devices that wasn't cutting it and that's why we were seeing those errors. I have changed this setting to allow up to 60 Megapixel images which should be fine unless you've got one of those wacky phones that allows 200MP images in which case turn that off because it's dumb.

Second, I have lowered the size that the system will resize images to. Originally, since I couldn't figure out the first problem, I set the system to resize to a max of 2000 pixels on the longest side with a max file size of 3MB. That adds up when you've got 48,000 images! I have reset this to 1600 pixels on the longest side with a max file size of 1.5MB.

To test this I made a 56MP image (10,000x5624 and 8MB) of my friend's Starfire VII and dragged it into this window. The system took it and resized it to this, which is 1600x900 and 259KB.



I certainly think that looks good enough for a website, but if we run into any issues with quality then we can revisit the settings and tweak accordingly.

In the future I may have the system convert images to webp, but that's not currently supported so it's not yet an option.

Anyway, that's what I did all day.

 

[GGJaguar]

I resize photos to make the file size smaller (between 150 and 450 KB) before I upload them. Are these still too large and do I need to make them smaller in future?

 

[shihan]

You are the mad professor. Thanks for the dedicated work.

 

[GAD]

I resize photos to make the file size smaller (between 150 and 450 KB) before I upload them. Are these still too large and do I need to make them smaller in future?

You shouldn't need to do that at all, but I appreciate your effort since your conversion is making smaller files than the system likely would which explains some of the numbers I'm seeing in regards to disk usage.

 

[JohnW63]

I always scale my files before upload and use 72dpi unless I really need more detail.

 

[GGJaguar]

You shouldn't need to do that at all, but I appreciate your effort since your conversion is making smaller files than the system likely would which explains some of the numbers I'm seeing in regards to disk usage.

Okay, thanks. I may reduce the file size a bit more as well as not posting as many pix to help preserve disk space.

 

[Canard]

Any preferences for graphics file format, JPEG, GIF, PNG?

 

[davismanLV]

Also, I'm not sure if this makes a difference, but when I quote someone's post for a reply, if the quoted post has an image or multiple images, I go into the quote and delete the images, leaving only the text. Does that make a difference? Or no?

 

[Cougar]

Thanks, GAD. You're a genius!

 

[GAD]

I always scale my files before upload and use 72dpi unless I really need more detail.

Thanks for the effort but DPI has no relevance to file size. It’s just a marker used for displays and printers. Resolution and compression are when drives file size.

Additionally the system will scale most images to the width of the post window unless over-ridden by the poster.

 

[GAD]

Any preferences for graphics file format, JPEG, GIF, PNG?

Nope. PNG is “newer and better” but all of them get compressed by the system so it doesn’t really matter in the end.

 

[GAD]

Also, I'm not sure if this makes a difference, but when I quote someone's post for a reply, if the quoted post has an image or multiple images, I go into the quote and delete the images, leaving only the text. Does that make a difference? Or no?

Not for disk space. Since the image is already on the system the copy you see isn’t taking up more space.

Deleting the copy does technically consume less network bandwidth, but the copy is smaller in the reply to alleviate that.

Again I appreciate the effort!

 

[GAD]

Thanks, GAD. You're a genius!

Aw, shucks.

 

[GAD]

Thanks for all the replies and for everyone trying to do the right thing!

Internet cloud systems are generally constrained and/or charged by the following:

CPU
Memory
Disk
Network bandwidth

Since I run Iinux, CPU and Memory is very well managed and is not an issue unless something goes wrong (think hackers or spammers).

Network bandwidth can be a real issue but with the system well tuned we’re in good shape there. Again the big risk here is hackers or spammers. If hackers breach the system and install a spam-bot then a month’s worth of network bandwidth can be consumed in hours. That generally also gets the server black-holed on email systems which is a royal PITA. You may recall this happening with the original software which led me to upgrade to the current software.

That leaves disk. Why is disk so expensive when you can buy huge drives on Amazon for cheap? Because the servers are all running SSD if not NVMe and that stuff is still very expensive.

I could run the system in my garage for cheap which I actually used to do before I owned LTG. Then Hurricane Sandy hit and we had no power and no Internet for 14 days. That was not fun. That’s when I learned that though I understand how to run a datacenter, I don’t actually *have* a datacenter, and that’s why LTG is in a VM in the cloud.

A hearty thanks to all the paid members because your membership almost completely pays for the server to run. In fact it was because if the paid membership that I was able to upgrade the disk space in the first place this allowing attachments to be enabled. My job is to make sure that space is being used efficiently so that another upgrade can be delayed for as long as possible.

 

[geoguy]

I've wondered if uploaded photos were problematic for you, GAD.

Would you prefer that members host their own photos (e.g., via imgur, etc.), and post links to those photos, instead of uploading files to LTG?

 

[GAD]

I've wondered if uploaded photos were problematic for you, GAD.

Would you prefer that members host their own photos (e.g., via imgur, etc.), and post links to those photos, instead of uploading files to LTG?

Nah. My goal with the site is first and foremost for it to be a repository of Guild knowledge. Hosting photos on-site ensures that threads don't break when other sites remove them (think the Photobucket debacle).

 

[AcornHouse]

I just uploaded my first pics since the change and I gotta say, Wow! So much faster.
Only crappy iPad pics, but got ’em on in a quarter of the time.

 

[chazmo]

@GAD, I first tried to upload a picture in Acrobat (PDF) format and got the size too big message. When I changed the format to JPEG the upload worked fine. Both images were around 2.3MB files...

I think I know why, but thought I'd check with you first... Is the reason the PDF won't upload because PDF format is not treated like a compressible picture by the software? If that's it, then I'll know not to use PDF format for this purpose.

 

[SFIV1967]

PDF isn't really a picture format.. PDF is a Portable Document Format...Use the Windows onboard Snipping Tool to snip the picture from your PDF if necessary.

Ralf

 

[chazmo]

PDF isn't really a picture format.. PDF is a Portable Document Format...Use the Windows onboard Snipping Tool to snip the picture from your PDF if necessary.

Ralf

Windows? What's that, Ralf?