CDN mapping like this:

http://cdn1.site.com|.jpg .jpeg .png .gif
http://cdn2.site.com|.jpg .jpeg .png .gif

And the CDN server could distribute every second JPG/PNG/GIF from the second CDN.

Alternatively allow path based mapping such as

http://cdn1.site.com|sites/default/themes|.jpg .jpeg .png .gif
http://cdn2.site.com|sites/default/files|.jpg .jpeg .png .gif

Please. :)

Files: 
CommentFileSizeAuthor
#25 cdn_single_img_cdn.png24.7 KBVacilando
#25 cdn_five_img_cdns.png19.98 KBVacilando
#25 cdn_five_img_cdns_subdomains.png2.8 KBVacilando
#17 1452092-17_D7.patch4.11 KBWim Leers
PASSED: [[SimpleTest]]: [MySQL] 16 pass(es).
[ View ]
#15 1452092-8_D7.patch4.1 KBWim Leers
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D7_1.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#13 1452092-8_D7.patch4.1 KBWim Leers
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D7_0.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#9 1452092-8_D6.patch2.37 KBWim Leers
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D6.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#8 1452092-8_D7.patch4.1 KBWim Leers
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D7.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#8 1417616-22_D6.patch5.49 KBWim Leers
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1417616-22_D6_0.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Comments

Title:Make it possible to map one filetype to multiple CDN'sAutomatically distribute certain filetypes over multiple CDNs
Component:Miscellaneous» Module
Assigned:Unassigned» Wim Leers
Priority:Normal» Major

I'd *swear* I had created an issue for this. I'm 100% certain I've written about this in an e-mail to someone or in some issue here. But I can't find it back.

Yes, I've been wanting to add this. However, I propose a different, clearer syntax, that will also make checking for this more efficient.

I propose this syntax:

http://cdn1.site.com http://cdn2.site.com|.jpg .jpeg .png .gif

It's consistent with what already exists. Thoughts? :)

Oh, FWIW, this is already possible currently: see the README. You can even configure it through the UI. It's just more involved than doing this automatically through the CDN mapping field, that's all!

Perhaps it's not available in the D6 version, but the CDN Mapping instructions are non-existant. It just says "Enter the domain name"

You should be seeing a message at the top, telling you to install the Advanced Help module if you want more detailed information. All information used to be in the descriptions of each form item. However, this was so overwhelming that this was actually annoying. Hence the Advanced Help module.

Found the post I was referring to in #1: http://drupal.org/node/1410318.

Quite right. Sorry for the noise.
Am actually interested in folder-designated server selection in the CDN Mapping textarea.

Folder-based CDN mapping will not be supported in the CDN mapping textarea. The majority of users don't need this. You're in fact the first user in years to request this.

However, you *can* implement that using the cdn_pick_server() function. Also see the README.

Status:Active» Needs review
StatusFileSize
new5.49 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1417616-22_D6_0.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
new4.1 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D7.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Patches for D6 & D7 attached. Please review.

Comes with updated unit tests: #1471472-1: Unit tests! :)

StatusFileSize
new2.37 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D6.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Wrong patch for D6 uploaded.

When this gets committed, I should also update the README.

Good to know folder based server picking (for theme vs upload and specific content type upload) will stay in cdn_pick_server().
I personally have had trouble getting that function to actually fire, but I'm sure it's just on my end.

I'm afraid I'm sure about that too :) You can always use the UI for that as well on the admin/config/development/cdn/other page.

StatusFileSize
new4.1 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D7_0.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Re-uploading the D7 patch of #8 (my god this is getting messy), so that the testbot will hopefully pick this up :)

StatusFileSize
new4.1 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1452092-8_D7_1.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Once more. I've followed the instructions at #1475086-1: Testbot not picking up contributed module patches?.

EDIT: HURRAY!

Status:Needs review» Needs work

The last submitted patch, 1452092-8_D7.patch, failed testing.

Status:Needs work» Needs review
StatusFileSize
new4.11 KB
PASSED: [[SimpleTest]]: [MySQL] 16 pass(es).
[ View ]

Reroll. Should apply now.

Note: because of the testbot, I can no longer upload D6 patches simultaneously, because the testbot doesn't look at filenames, only at the version indicated in the issue. Asked for help in #1475256: Development in 2 branches simultaneously; patches posted for both too: how to make testbot deal with this appropriately?.

HURRAY! :D :) :) :)

*Now* we're rolling!

If @doublejosh could now test and review it too, then I can commit this. This would be the first commit for the 2.5 release :)

#9 is the correct one for D6 yes?
I'm actually just interested in folder splitting, but am working on CDN right now so can do some testing next work week.

Yes. Sorry for the mess — not that it's really my fault, but anyway :)

Ran the patch after updating to 6.x-2.4 with a fail on one portion...

$ patch < 1452092-8_D6.patch
patching file cdn.basic.inc
Hunk #1 FAILED at 96.
1 out of 1 hunk FAILED -- saving rejects to file cdn.basic.inc.rej
patching file cdn.module
Hunk #1 succeeded at 160 (offset -17 lines).

Thanks for the tips, managed to get the balancing between 2 servers to work so now it's no longer 90% CDN1 and 10% CDN2 due jpegs :)

@doublejosh: can you do a reroll of the patch? Recent commits have broken it. The D7 patch should still apply.

Is the specific file and cdn server link maintained between page loads? E.g. if certain image is uploaded to one of the CDNs, will it be pulled from there also next time, or can it happen that next time it will be uploaded and served from another CDN?
If the link is not maintained, doesn't it mean that all files will eventually be (possibly in different versions) copied in CDN_1-CDN_N (resulting in N x the size of CDN space needed). What's the strategy here?

Status:Needs review» Reviewed & tested by the community
StatusFileSize
new2.8 KB
new19.98 KB
new24.7 KB

Works flawlessly! See a screenshot of waterfall for one local image subdomain, then two screenshots where images are distributed over 5 image subdomains.

You can see that all images load in parallel, which has been the objective. You can also see that they are called randomly.

I suppose the patch (D7 at least) can now be submitted to dev.

Questions:

  1. See #24.
  2. I wonder why the actual loading time, in the parallel download case, is not much shorter. Seems that the browser is waiting there, so would this be solved by setting Apache's MaxClients higher? Any ideas here?

#24: yes, the "link" is maintained.
#25: Great :) The tests say it should work flawlessly, so I'm glad the reality confirms that :) As to why it's not faster: was this the first page load? The CDN (assuming you're using an Origin Pull CDN) has to cache it first, so that may be the reason.

I'll commit this soon then, thanks for the review!

Cool, looking forward to the new dev!

It's like that for the first and subsequent pageloads. Origin pull - but from a fake CDN (local subdomain) -- perhaps that's the reason. I should open another issue to clarify this because it happens independently from the automatic distribution over CDNs.

This happens independently from that. It only depends on the filename you feed it. You can easily see that in the source code too, FYI.

Status:Reviewed & tested by the community» Fixed

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Thanks for this! It's really simple, like the rest of the module.

I have a view with 10 images on each page. On most pages 3 images get served from the first server and 7 from the second one (on some pages it's 5/5, on some 2/8). Is this intended? It's actually good in my case. I'd like to stay it that way. But I expected about half of the files to be served from each server.

#32: this issue was closed a few hours before you commented on it. Hence nobody noticed your comment.

The CDN module does not try to perfectly balance files. It could, but then it could happen that image A is served from CDN X on page 1 and from CDN Y on page 2. Instead of providing perfect balancing, we guarantee that image A is always served from CDN X.
However, if you really would like perfect balancing, then you should open a new issue for that and provide a patch. :)