Lance Cleveland

Cleaning Up WordPress – Large wp_options.idb

Did your tiny AWS Bitnami WordPress server run out of disk space? The culprit may be unoptimized data tables in MySQL. Finding a 4.9GB wp_options.idb file on your server is not as uncommon as you may think. What makes it surprising is when you look at your wp_options entries and discover there are only 300 rows in that table with limited text in the option_value column.

Thankfully there is an easy fix as long as you can get enough disk space to manage the task. Start by looking for any log files or other files you are CERTAIN you don’t need so you can shut down web, php, and mysql services.

Finding Large Files

# sudo find / -xdev -type f -size +50M -print | sudo xargs ls -lh | sudo sort -k5,5 -h -r

This command finds all files on the drive over 50M in size.

Look for specific MySQL raw data files (wp_options.idb for example) in that list. Remember the table name for later.

Stopping System Services

Stop the services. On an AWS Bitnami WP server they have their own special control scripts in place of the standard Linux services library.

bitnami@ip-172-31-87-127:/opt/bitnami/mysql$ sudo /opt/bitnami/ctlscript.sh stop mysql


usage: /opt/bitnami/ctlscript.sh help
       /opt/bitnami/ctlscript.sh (start|stop|restart|status)
       /opt/bitnami/ctlscript.sh (start|stop|restart|status) mysql
       /opt/bitnami/ctlscript.sh (start|stop|restart|status) php-fpm
       /opt/bitnami/ctlscript.sh (start|stop|restart|status) apache

help       - this screen
start      - start the service(s)
stop       - stop  the service(s)
restart    - restart or start the service(s)
status     - show the status of the service(s)

Stop all the services, then start MySQL only.

Cleaning Up MySQL

If you are doing system admin commands you should know how to find your data access credentials in the wp-config.php file. No need to go into that here. Find the credentials and login to MySQL.

No OPTIMIZE that table (and any others that you suspect have unusually large .idb files).

mysql> OPTIMIZE NO_WRITE_TO_BINLOG TABLE wp_options;

That’s it. One simple command may easily recover 4.9GB of disk space , making a 4.9GB file on a 9GB drive get back to a more reasonable 10MB.

Why WordPress core or cron jobs are not doing this on a regular basis is a question for another day…

Hate Spam? Turn Off Jetpack Email Sharing

jetpack publicize spam

The past few days have been spent diagnosing various email delivery issues from the AWS web cluster that is running our WordPress plugin store as well as our SaaS locator platform. During this process email routing was pushed from the servers through the AWS Simple Email System. SNS notifications were enabled to monitor the progress and provide some insight as to what was happening on the send mail side of things.

Not far into the mission something odd was showing up. Email delivery notifications were being transmitted from our documentation server — a basic WordPress install with almost no plugins running and a simplified theme. Yet in the “delivered” stack of SNS notifications there were random email addresses being spammed every 30-60 seconds.

The Culprit? Jetpack

Turns out the documentation site has Jetpack installed. It also has the default settings for the publicize sharing enabled. This includes email sharing.

After a good bit of research it was found that the mailing subsytem was being exploited through the front end interface for sharing a post via email. The Postfix mail logs provide immediate evidence of this.

Read More

AWS LEMP Stacks and EFS Issues

Lesson learned — if you are using EFS on production systems you want to be using provisioned throughput mode.

But, before we get into that, let’s go over the details of this implementation…

Service Configuration

We utilize AWS EC2 instances to run multiple WordPress sites hosted in different directories. The configuration is fairly standard: 2+ servers configured as part of an load-balanced cluster. The servers run from the same image meaning they use the same underlying software stack.

Part of that image includes a mounted EFS (Elastic File Storage) directory , used to share WordPress resources between all nodes in the cluster. The original architecture was designed to host not only the typically-shared wp-content/uploads folder of WordPress via this EFS mount but also the code. The thought was that sharing the code in this way would allow a system admin to easily update WordPress core, plugins, or themes from the typical wp-admin web login. Any code updates would immediately be reflected across all nodes.

EFS Web App Code Hosting – A Bad Idea

Turns out this is a bad idea for a few reasons. First of all, EFS volumes are mounted using the NFS4 (network file storage) protocol — this defines how the operating system handles file read/write operations for a network mounted drive. While NFS4 is fairly robust, the throughput of ANY network drive, even on a high speed AWS data center backbone, is much slower than a local drive such as an EBS volume.

That means that even on a good day every PHP file, JavaScript file, or anything else needed to serve up that WordPress web page are going to be a bit slower than normal.

However, the bigger problem comes to light if you happen to choose the default, and pushed as “the mode to use” by Amazon, EFS throughput mode known as “Burst mode”.

Read More

Internet Enabled Voting For US Elections

Internet enabled voting is a must IMO. Many that are afraid of the security risks and remote hacking have a very shallow, if any, understanding of the risks involved.

To claim physical (paper) voting is more secure is absurd. Every country that has used that system, including ours, has encountered fraud in some form.

Maybe this is the perfect catalyst for getting our Internet providers to finally enable IPV6. It would make external attacks a couple orders of magnitude more difficult. Not too mention providing direct 1:1 accountability to track every single device used to vote.

https://www.npr.org/2019/11/07/776403310/in-2020-some-americans-will-vote-on-their-phones-is-that-the-future

Internet Enabled Voting For US Elections

apple.news/A2hlUbDHwRjifTbSwv6fOZQ

Internet enabled voting is a must IMO. Many that are afraid of the security risks and remote hacking have a very shallow, if any, understanding of the risks involved.

To claim physical (paper) voting is more secure is absurd. Every country that has used that system, including ours, has encountered fraud in some form.

Maybe this is the perfect catalyst for getting our Internet providers to finally enable IPV6. It would make external attacks a couple orders of magnitude more difficult. Not too mention providing direct 1:1 accountability to track every single device used to vote.

%d bloggers like this: