AUTOMATIC static mirroring?

[Sapphireblue]

Hello all---

I am in need of a module to do static mirroring of a Miva Merchant 4.18/OUI site. Not for search engine purposes so much, but so that another website can scrape product and category pages from the Merchant store with minimal impact on the Merchant server.

SEK and Weiland's Static Catalog Generator seem to have vanished into thin air, so as far as I can tell, that leaves Phosphor Media's Merchant Optimizer. It looks like it'd suit our needs, except for one thing---it looks like mirroring needs to be manually invoked. Is that true? What I really need is for a static mirror to be created, say, once a night at 3 a.m. Is there any way to accomplish that, with Optimizer or with some other tool? Thanks very much.


m

[Vic - WolfPaw Computers]

In this instance, why not upgrade to MM5 with the MySQL backend?

Perfect for this kind of application. You can use PHP to pull data direct from the MySQL tables to display inside any look & feel you'd like with the PHP front end.

[Sapphireblue]

If I were building a brand-new website + Merchant store today, that's how I'd do it. However, I'm working with a site and a store that are circa 2003ish: old enough to be behind the technological curve, but new enough that my client would have no interest in spending a wad of money to port it over. And, since the existing non-Merchant site is ColdFusion/SQL Server, it'd be quite a wad indeed.

[dotCOM_host]

Not with Merchant Optimizer by itself (GREAT tool, btw!) but check out MmTicker from truXoft. I'm not 100% sure it will work, but it was designed to handle a lot of the tasks automatically and knowing Ivo - I'm sure he'd find a way to make MmTicker do exactly what you need. He's on this forum so check with him if MmTicker can be tweaked to do this task for you on a nightly basis.

[ILoveHostasaurus]

Hello all---

I am in need of a module to do static mirroring of a Miva Merchant 4.18/OUI site. Not for search engine purposes so much, but so that another website can scrape product and category pages from the Merchant store with minimal impact on the Merchant server.

SEK and Weiland's Static Catalog Generator seem to have vanished into thin air, so as far as I can tell, that leaves Phosphor Media's Merchant Optimizer. It looks like it'd suit our needs, except for one thing---it looks like mirroring needs to be manually invoked. Is that true? What I really need is for a static mirror to be created, say, once a night at 3 a.m. Is there any way to accomplish that, with Optimizer or with some other tool? Thanks very much.


Optimizer is smart enough to keep track of changes made in the admin so it knows which pages need to be regenerated. There would normally be no reason to regenerate the entire site every day because the data would not have changed. If someone is changing the data, they just need to remember to click the Optimizer tab before leaving for the day and it will tell them which pages need to be updated and will do it for them.

[dotCOM_host]

If someone is changing the data, they just need to remember to click the Optimizer tab before leaving for the day and it will tell them which pages need to be updated and will do it for them.
That would work only half-way... When you delete products from your store, you need to re-generate the category pages these products belong to, so they can be removed from them, as well as any links from any other pages that may be linking to these products. Merchant Optimizer, as great as it is, doesn't have the knowledge to update the parent categories automatically for you... What you described would work great if you are simply updating existing products, in which case yes, MO will regenerate only the product html pages that have changed, but if you add or delete products, you pretty much need to re-run the whole thing, or at least the categories these products belong to.

[Sapphireblue]

I'm trying to avoid my client having to click a button to generate her static mirror, trying to avoid adding an extra step to her process of adding/updating products and categories. I know it's very little to ask of her really, but it's a more reliable solution if I don't have to depend on some human to get it done. And realistically?---her data files don't get packed, her abandoned carts don't get deleted---if the mirror process isn't automatic (whether scheduled or realtime), the mirror will slowly but surely go out of date.

As a developer, you know, there's always the "well, I told her, so it's not my problem" route, but I'm hoping I can do better than that.

Remik: that is an interesting idea... but do I have to turn in my geek card if I say Ivo's modules scare me? One of 'em bit me once and now I'm a little flinchy. Of course I will have to get over that in a hurry if none of my other ideas pan out.

[dotCOM_host]

Remik: that is an interesting idea... but do I have to turn in my geek card if I say Ivo's modules scare me? One of 'em bit me once and now I'm a little flinchy. Of course I will have to get over that in a hurry if none of my other ideas pan out.
:-) Ivo's modules are only "scary" because they tend to have a lot of features in them and it may take more than 5 minutes to learn how to use them. I can't fault him for it though - as you well know, it's next to impossible to please everyone with one set of predefined settings, so he goes overboard on making his modules very flexible and adjustable to just about any scenario. Don't be too afraid of his modules - they are not all THAT scary, and once you get a hang of them, they are actually quite nice and very powerful.

Since you mentioned the need to keep the store owner away from the routine tasks that they "should be" doing but aren't... that's one more reason to have MmTicker in their store. It most certainly can delete expired baskets and pack databases on a regular schedule without any user interaction. Definitely have a look at it and check with Ivo to see if it can be tweaked to run Merchant Optimizer for you every night. You'll kill two problems with this one module - and your client should love you for it. :)

[ILoveHostasaurus]

Well that's easy enough to work around, you can pass everything you need to pass as a regular URL:


https://www.domain.com/Merchant2/admin.mvc?Screen=SUTL&Action=SUTL&Store_Code=ABCD&pm_static_action_cats=All&Module_Code=pm_static&UserName=user&Password=pass
https://www.domain.com/Merchant2/admin.mvc?Screen=SUTL&Action=SUTL&Store_Code=ABCD&pm_static_action_prods=All&Module_Code=pm_static&UserName=user&Password=pass


Those two URL's will cause an Optimizer store to regenerate all categories and all products. You just have to put in the appropriate domain name, appropriate store code and a username and password that have permissions to run Optimizer.

The two downsides:

1) Make sure your webstats are protected since you're going to be passing a username and password in the URL and if you request it often enough, it will probably end up in your stats.

2) Because Optimizer normally runs for longer than most Empresa timeout values, it uses meta refreshes in the resulting html to continue it running around the Empresa timeouts. That means there is no way to make it regenerate the pages by calling those above URL's from a text-based web browser, since they won't keep following the refreshes.

Maybe Bruce could give Optimizer the option to issue HTTP 302 redirects instead of meta refreshes. So it would start, do some work, issue a 302 redirect to the next page, do more work, 302, etc. Not sure how browsers would handle being given a series of redirects. If that did work, then all you need is a cron job that makes the regeneration request each night.

[Bruce - PhosphorMedia]

Interesting idea...we will take a look at this...been meaning to add this feature for a while...but have been busy re-writing the entire program for M5 (which should have this feature as well as a boatload of others....and its pretty close...but, I degress)...

[ILoveHostasaurus]

Just be careful to not output any data whatsoever before the redirect, otherwise it will fail.

[Sapphireblue]

Huh. Remik, I think that was a "yes" on the geek card question? :D

The scary is because one of Ivo's modules actually broke really ugly on me during the install process while I was developing this store, about 2.5 years ago. It's possible, even likely, that it happened while I was at my old crappy Merchant host (who could barely keep their servers running), and that it was 100% the host's fault, not the module's. Whatever the cause, the result wasn't pretty, hence the flinchiness.

Still, I understand that Ivo has a lot of very satisfied customers and that his modules are a power-user's dream, so it's worth considering I guess.

Meanwhile, perhaps Phosphor will get the email I sent them earlier today and think of it as an enhancement request. I hadn't realized you could pass Admin u/p through a URL, so that's almost the perfect answer for a scheduled-task type scenario... except for the meta refresh issue David points out. So close, yet so far! If I hear back from Phosphor I might ask them about that 302. Will report back here too, for the benefit of future archives searchers :D

[Sapphireblue]

aaaand, there he is. The circle is complete.