问题描述
通常建议通过 startrow 和 stoprow 使用范围扫描，而不是使用 Rowkey前缀过滤器（例如，）。原因在于 Rowkey前缀过滤器导致rowkey的全表扫描，而通过 startrow 和 stoprow 不会导致全表扫描。为什么不呢？大多数人说因为rowkey是按照词汇顺序存储的，这当然并不能解释为什么 Rowkey前缀过滤器不能利用这一点。
无论如何，通过 startrow 和 stoprow 进行范围扫描的结果不是结果在rowkey的全表扫描中？
以python中的这个小例子来说明为什么我不明白rowkeys的lexagraphical顺序是什么意思以避免全表扫描：
rowkeys = ['a1'，'a2'，'a3'，'b1'， 'b2'，'b3'，'c1'，'c2'，'c3']
def range_scan（startrow，stoprow）：
is_found = False
for rowkey in rowkeys：
if startrow< = rowkey< stoprow：
is_found = True
产生rowkey
else：
如果is_found：
提升StopIteration（）
显然，HBase算法与此不同。它是如何的？
TLDR：当使用startrow和stoprow进行范围扫描时，HBase如何避免全表扫描？
解决方案在HBase中，表格被分割成区域。区域是由特定区域服务器提供的一组行。一个区域服务器（通常）有多个表中的多个区域来处理请求。
因为行按键排序，所以在hbase master中是已知的启动和停止键是每个区域的边界以及该区域可以在哪个区域服务器上进行查询。因此，如果扫描完成后使用明确的开始和停止那么该查询将仅指向具有可能在请求范围内的键的区域/区域服务器。
如果查询具有小范围的键那么你会发现在几乎所有的查询中，只有一个区域被访问。
HBase表格有很多区域，并且使用start和停止行以限制扫描，您可能会发现表中100个区域中的99个甚至不知道表上的查询已完成。
It is commonly suggested to use a range scan via startrow and stoprow as opposed to a Rowkey Prefix Filter (for example, here). The reasoning for this is because a Rowkey Prefix Filter results in a full table scan of the rowkey, whereas a range scan via startrow and stoprow do not result in a full table scan. Why doesn't it? Most people say "because the rowkey is stored in lexographical order," which of course, doesn't explain why the Rowkey Prefix Filter cannot leverage this.
At anyrate, how exactly does a range scan via startrow and stoprow not result in a full table scan of the rowkey?
Take this small example in python to show why I don't understand how the lexagraphical ordering of the rowkeys means anything when it comes to avoiding a full table scan:
rowkeys = ['a1', 'a2', 'a3', 'b1', 'b2', 'b3', 'c1', 'c2', 'c3']
def range_scan(startrow, stoprow):
is_found = False
for rowkey in rowkeys:
if startrow <= rowkey < stoprow:
is_found = True
yield rowkey
else:
if is_found:
raise StopIteration()
Clearly, the HBase algorithm differs from this. How does it?
TLDR: How exactly does HBase avoid a full table scan when doing a range scan with startrow and stoprow?
解决方案 In HBase a table is split into regions. A region is a set of rows that is served by a specific region server. A region server has (in general) multiple regions from multiple tables for which it handles the requests.
Because the rows are sorted by key it is known in the hbase master which start and stop keys are the boundaries of a each region AND on which region server this region can be queried.
So if a scan is done that uses an explicit start and stop row then the query is directed ONLY to the regions/region servers that have keys that may be in the requested range.
If the query has a 'small' range of keys then you'll find that in almost all queries only a single region is accessed.
It is common to have dozens of regions for a HBase table and by using the start and stop row to limit the scan you may find that 99 of the 100 regions of your table do not even know a query on the table has been done.
这篇关于如何使用startrow和stoprow不会导致HBase进行全表扫描？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！