简单分片逻辑

news/2024/7/19 13:46:05 标签: java, js, 大数据, 设计模式, web

Sharding is the technique where you split static components across multiple domains so the browser can fetch more components in parallel? (Which browser? How many components per hostname? Ask browserscope.org)

分片技术是将静态组件拆分到多个域中,以便浏览器可以并行获取更多组件的技术吗? (哪个浏览器?每个主机名有多少个组件?请询问browserscope.org )

So you have, say, image-a.png and image-b.png. You want to serve image-a from domain1.example.org and image-b.png from domain2.example.org

因此,您有image-a.png和image-b.png。 您要提供来自domain1.example.org的image-a和来自domain2.example.org的image-b.png

The thing is you don't want to randomize which image goes to which host in different page views. Otherwise you'll do like cnn.com where the same image (a 1x1 gif) is served from three different domains (in the same pageview even) causing three HTTP requests.

问题是您不想在不同页面视图中随机分配哪个图像到哪个主机。 否则,您会像cnn.com一样,其中从三个不同的域(甚至在同一浏览量中)提供相同的图像(1x1 gif),从而导致三个HTTP请求。

Here's a simple solution: use modulus to split components into buckets based on their URL (or path name).

这是一个简单的解决方案:使用模数将组件根据其URL(或路径名)拆分为存储桶。

Billy "Zoompf" Hoffman mentioned this as a simple strategy - if you want to split to two hostnames, take the length of the image path. If it divides by two - serve from host 2. Otherwise - host 1.

Billy“ Zoompf” Hoffman提到这是一个简单的策略-如果您想拆分为两个主机名,请使用图像路径的长度。 如果除以2,则从主机2服务。否则,从主机1服务。

Same simple logic can apply for splitting into as many hostname buckets as you want. Need three buckets? % 3. Four buckets? % 4.

可以使用相同的简单逻辑将主机名存储桶拆分为任意数量。 需要三个水桶吗? %3.四个水桶? %4。

You can even do it per browser, for example if it's IE6, split to more buckets.

您甚至可以针对每个浏览器执行此操作,例如,如果是IE6,则拆分为更多存储桶。

Here's the SSL (Simple Sharding Logic 🙂 ) expressed in JavaScript:

这是用JavaScript表示的SSL(简单分片逻辑):

function getBucket(url, numbuckets) {
  var number = url.length,
       group = number % numbuckets;
  return group;
}

If you think most of your paths will have same length, e.g. /images/top.png, /images/nav.png, /images/bot.png you can use some other number, e.g. the character code of the middle letter in the path.

如果您认为您的大多数路径都具有相同的长度,例如/images/top.png、/images/nav.png、/images/bot.png,则可以使用其他一些数字,例如中间字母的字符代码路径。

var number = url.charCodeAt(parseInt(url.length/2));

Or the code of the letter in 1/4 of the path + the one in the middle + the on in 3/4 of the path. You get the point - all you need is a number that can be produced from the file path (or content or anything) and won't change from one page view to the next. You need stable file-to-hostname resolutions above all.

或路径的1/4中的字母代码+中间的一个+路径3/4中的on的代码。 您明白了-您所需要的只是一个可以从文件路径(或内容或其他任何内容)生成的数字,并且不会从一个页面视图切换到另一个页面视图。 首先,您需要稳定的文件到主机名解析。

You can also pass an array of components and get a multi-array of the components, grouped into buckets. Here's what I tried on cnn.com and worked beautifully:

您还可以传递一组组件,并获得组成该组的多个组件组。 这是我在cnn.com上尝试并做得很漂亮的方法:

function toBuckets(stuff, numbuckets) {
  var numbuckets = parseInt(numbuckets, 10),
      url, group,
      buckets = Array(numbuckets),
      cache = {};
  for (var i = 0, max = stuff.length; i < max; i++) {
    url = stuff[i].src;
 
    if (typeof cache[url] === 'number') {
      continue;
    } 
    group = getBucket(url, numbuckets);
    if (!buckets[group]) {
      buckets[group] = [];
    }
    buckets[group].push(url);
    cache[url] = group;
  }
  return buckets;
}
 
console.log(toBuckets(document.images, 3));

This gave me (on cnn.com) a nice list of three buckets and the URLs in each:

这给了我(在cnn.com上)一个不错的清单,其中包含三个存储桶以及每个存储桶中的URL:

[
  ["http://i2.cdn.turner.co...seals.02.cnn%5B1%5D.jpg", 
   "http://i2.cdn.turner.co...ves/tzvids.osama.gi.jpg", 
   "http://i.cdn.turner.com.../misc/advertisement.gif", 5 more...], 
  ["http://i.cdn.turner.com...der/hat/arrow_black.png",
   "http://i2.cdn.turner.co...ds.military.dogs.gi.jpg",
   "http://i2.cdn.turner.co...bl.house.cnn.120x68.jpg", 8 more...], 
  ["http://i.cdn.turner.com...element/img/3.0/1px.gif",
   "http://i.cdn.turner.com...bal/header/hdr-main.gif", 
   "http://i.cdn.turner.com...al/header/nav-arrow.gif", 9 more...]
]

Not too bad for such simplicity.

对于这样的简单来说还不错。

As you see, I have a cache object, so it takes care of any duplicates so I don't end up with the same component repeating.

如您所见,我有一个缓存对象,因此它可以处理所有重复项,因此最终不会重复相同的组件。

The drawback of course is that it's unlikely that you'll get exactly the same number of components in each bucket. But from what I tried on a few sites, that's not a big issue. The benefit of having a stable and simple (no databases, cookies, local storage, etc) resolution of file path to a hostname is a win.

当然,缺点是在每个存储桶中获得的组件数量不可能完全相同。 但是从我在几个站点上尝试过的结果来看,这并不是一个大问题。 拥有稳定且简单的主机名文件路径解析(无数据库,Cookie,本地存储等)的好处是双赢。

Tell your friends about this post on Facebook and Twitter

在Facebook和Twitter上告诉您的朋友有关此帖子的信息

翻译自: https://www.phpied.com/simple-sharding-logic/


http://www.niftyadmin.cn/n/1306032.html

相关文章

JavaScript控制输入框只能输入非负正整数

1、问题背景 问题&#xff1a;一个输入框&#xff0c;输入的是月份&#xff0c;保证输入的内容只能是非负正整数 2、JavaScript代码 function checkMonth() { $("month").keyup(function(){ var temp $(this).val(); $(this).val(temp.replace(/ \ D/^0/g,)); })b…

org.hibernate.MappingException

1、错误描述 org.springframework.beans.factory.BeanCreationException:Error creating bean with name sessionFactory defined in ServletContext resource [/WEB-INF/applicationContext.xml]:Invocation of init method failed;nested exception is org.hibernate.Mappin…

E: 无法获得锁 /var/lib/apt/lists/lock - open (11: 资源暂时不可用)

1 错误描述 youhaidongyouhaidong:~$ sudo apt-get update E: 无法获得锁 /var/lib/apt/lists/lock - open (11: 资源暂时不可用) E: 无法对目录 /var/lib/apt/lists/ 加锁 2 错误原因 权限不够&#xff0c;导致出错 3 解决办法 youhaidongyouhaidong:~$sudo rm /var/lib/apt…

预加载,然后执行

I talked before about using an object element to load scripts and styles without executing/applying them to the current document. And this is nice for preloading assets - faster and less error-prone than simple inclusion if the document or iframe. 我之前讨…

E: 未发现软件包 install_flash_player_11_linux.x86_64.tar.gz

1 错误描述 youhaidongyouhaidong:~$ sudo apt-get install install_flash_player_11_linux.x86_64.tar.gz 正在读取软件包列表... 完成 正在分析软件包的依赖关系树 正在读取状态信息... 完成 E: 未发现软件包 install_flash_player_11_linux.x86_64.tar.gz E: 无法按照正则…

CSS铁路图

So next step after the sexy CSS lexer is parsing. But first - railroad diagrams to help visualize how/when the tokens make sense forming valid CSS code. 因此&#xff0c;在解析性感CSS词法分析器之后的下一步。 但是首先-铁路图有助于可视化令牌形成有效CSS代码的方…

tar (child): jdk-7u71-linux-x64.tar.gz:无法 open: 没有那个文件或目录

1 错误描述 youhaidongyouhaidong:~$ sudo mkdir /usr/lib/jvm [sudo] password for youhaidong: youhaidongyouhaidong:~$ sudo tar zxvf jdk-7u71-linux-x64.tar.gz -C /usr/lib/jvm tar (child): jdk-7u71-linux-x64.tar.gz&#xff1a;无法 open: 没有那个文件或目录 tar…

圆角矩形_圆角

圆角矩形Warning: not practical blog post, dont read, move on. 警告&#xff1a;不实用的博客文章&#xff0c;请勿阅读&#xff0c;继续前进。 So this is a post about a thought I had - creating rounded corners in IE678 by using roundness that they already have b…