For those who are not aware, PageLines.com’s static site files were deleted from the system not once, but twice on launch day. We’ve been working at getting everything back to where it’s supposed to be; and I’ll write an update here.
First, just wanted to apologize for these issues. We’re not as responsive as we usually are, and we really REALLY would like to get back to creating AMAZING experiences for PageLines users.
The Systems We’re Working On…
Here are the main system that are down and the timeline to getting things back…
- DMS Pro Activations:
Some users are experiencing issues with activations. To our knowledge 80% of these should be fixed now. Any lingering issues will be dealt with by end of weekend.
The PageLines forums are going to be moved to: http://forum.pagelines.com and should be back up by tomorrow morning.
- Legacy “Launchpad” Customer System:
This will be moved to http://launchpad.pagelines.com and should be back up for you by tomorrow morning.
- Upgrading Grandfathered Plus Subscribers:
Working on this as well, everyone who was on Plus will be grandfathered to DMS Pro Business or Developer by Monday.
- Forum is live at http://forum.pagelines.com
- Products and PL Framework are available via Launchpad which is back up at http://launchpad.pagelines.com
By End of Week:
- Fix on v2 Store API; which will repair v2 store, other issues.
- v2 Store System,
- Finishing up grandfathering Plus members into DMS Pro
At around 11pm PST on Tuesday, as we were preparing to launch, someone noticed that PageLines.com was down.
When we went into the system to figure it out something was definitely wrong… Somehow all static files on the server had been deleted.
At this point, we suspected a hack but weren’t sure. PageLines has a great backup system and so the process of restoring the site began. By about midnight PST the team had reoriented itself and discovered that some scripts and files were lost and would need to be rewritten.
By 4am PST the site had been fully restored to launch-ready status.
The team began to consider the possibilities as to what had happened and decided that *if* the server had been compromised, restoring a back up would not fully resolve the issue as that vulnerability would be in the backup. But it was the fastest way to get back up and running.
However, we did suspect that if it *was* a hack, the hacker would not be satisfied with how fast we’d recovered and would do it again in the morning.
With the official Launch scheduled for 10:30am, at 10:21am, the site was compromised and all static files were lost for a second time.
Time for a new server
The server switch is where the issues come in, since everything has to be reconfigured and reinstalled.
It’s important to note here that PageLines takes security very seriously. Especially the security of users’ personal information. None of that information was compromised.
How it happened?
If we were hacked, it has nothing to do with PageLines software and probably not WordPress either.
Most likely someone was able to gain some access to the filesystem through a backdoor in a WordPress plugin we were using. They had probably figured this out over the past weeks, and were biding their time until we were close to launch.
Backdoors can be tough to find and are often well disguised. For this reason we have disabled most plugins, and are checking these systems.
Thanks for your patience and support
We want to say “thanks” to the community including PageLines developers, enthusiasts and especially the members of the ‘300’ who have supported us through this.
You guys are the reason we work so hard and your support in social media and blog post comments helped tremendously.