Changes between Version 3 and Version 4 of waue/2009/0609

Timestamp:

Jun 9, 2009, 4:22:47 PM (15 years ago)

Author:

waue

Comment:

--

waue/2009/0609

@@ -13 +13 @@
 
 09/06/09 12:18:13 INFO mapred.MapTask: data buffer = 79691776/99614720
+
 09/06/09 12:18:13 INFO mapred.MapTask: record buffer = 262144/327680
+
 09/06/09 12:18:14 INFO crawl.CrawlDbReader: TOTAL urls: 1072
 09/06/09 12:18:14 INFO crawl.CrawlDbReader: status 1 (db_unfetched):    1002
+
 09/06/09 12:18:14 INFO crawl.CrawlDbReader: status 2 (db_fetched):      68
+
 }}}
  === convdb ===
+ - 沒啥用
+ === inject ===
+ 
+{{{
+
+}}}
+ === readlinkdb ===
+{{{
+
+}}}
 
  ===  ===
+{{{
 
- ===  ===
+}}}
+ === readseg ===
+ - read / dump segment data
+ - Usage: SegmentReader (-dump ... | -list ... | -get ...) [general options]
+ - SegmentReader -dump <segment_dir> <output> [general options]
+ - SegmentReader -list (<segment_dir1> ... | -dir <segments>) [general options]
+ - SegmentReader -get <segment_dir> <keyValue> [general options]
+{{{
 
- ===  ===
+}}}
+ === updatedb ===
+ - update crawl db from segments after fetching
+ - Usage: CrawlDb <crawldb> (-dir <segments> | <seg1> <seg2> ...) [-force] [-normalize] [-filter] [-noAdditions]
+{{{
+$ nutch updatedb /tmp/search/crawldb/ -dir /tmp/search/segments/
+}}}
+ === dedup ===
+ - remove duplicates from a set of segment indexes
+ - Usage: DeleteDuplicates <indexes> ...
+{{{
+$ nutch dedup /tmp/search/indexes/
 
- ===  ===
-
- ===  ===
-
- ===  ===
-
+}}}
  == 筆記 ==