博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
python mongodb MapReduce
阅读量:6829 次
发布时间:2019-06-26

本文共 3174 字,大约阅读时间需要 10 分钟。

# -*- coding: utf-8 -*-

import os
import csv
import pymongo
from pymongo import MongoClient
from bson.code import Code
from pymongo import MongoClient

 

#建立连接

client = MongoClient('10.20.4.79', 27017)
#client = MongoClient('10.20.66.106', 27017)
db_name = 'ta' #数据库名
db = client[db_name]

 

插入测试数据:

  for i in xrange(1000):

    rID=math.floor(random.random()*10); 
    price = round(random.random()*10,2); 
    if rID < 4:
      db.test.insert({"_id":i,"user":"Joe","product":rID,"price":price}); 
    elif rID>=4 and rID<7:
      db.test.insert({"_id":i,"user":"Josh","product":rID,"price":price}); 
    else:
      db.test.insert({"_id":i,"user":"Ken","product":rID,"price":price});

  结果数据为: 

  { "_id" : 0, "price" : 5.9, "product" : 9, "user" : "Ken" }

  { "_id" : 1, "price" : 7.59, "product" : 7, "user" : "Ken" }
  { "_id" : 2, "price" : 4.72, "product" : 0, "user" : "Joe" }
  { "_id" : 3, "price" : 1.35, "product" : 1, "user" : "Joe" }
  { "_id" : 4, "price" : 2.31, "product" : 0, "user" : "Joe" }
  { "_id" : 5, "price" : 5.29, "product" : 5, "user" : "Josh" }
  { "_id" : 6, "price" : 3.34, "product" : 1, "user" : "Joe" }
  { "_id" : 7, "price" : 7.2, "product" : 4, "user" : "Josh" }
  { "_id" : 8, "price" : 8.1, "product" : 6, "user" : "Josh" }
  { "_id" : 9, "price" : 2.57, "product" : 3, "user" : "Joe" }
  { "_id" : 10, "price" : 0.54, "product" : 2, "user" : "Joe" }
  { "_id" : 11, "price" : 0.66, "product" : 1, "user" : "Joe" }
  { "_id" : 12, "price" : 5.51, "product" : 1, "user" : "Joe" }
  { "_id" : 13, "price" : 3.74, "product" : 6, "user" : "Josh" }
  { "_id" : 14, "price" : 4.82, "product" : 0, "user" : "Joe" }
  { "_id" : 15, "price" : 9.79, "product" : 3, "user" : "Joe" }
  { "_id" : 16, "price" : 9.6, "product" : 5, "user" : "Josh" }
  { "_id" : 17, "price" : 4.06, "product" : 7, "user" : "Ken" }
  { "_id" : 18, "price" : 1.37, "product" : 5, "user" : "Josh" }
  { "_id" : 19, "price" : 6.77, "product" : 9, "user" : "Ken" }

 

测试1、每个用户各购买了多少个产品? 用SQL语句实现为:select user,count(product) from test group by user  mapper = Code("""function (){emit(this.user,{count:1})}""")

  reduce = Code("function (key, values) {"

    " var total = 0;"
    " for (var i = 0; i < values.length; i++) {"
    " total += values[i].count;"
    " }"
    " return {count:total};"
    "}")

    result=db.test.map_reduce(mapper,reduce,out ='myresults')

  for doc in db.myresults.find():

    print doc

   测试 2、查询每个用户,买了多少商品,总价格,及评价价格   条件是价格大于5的 SQL实现:select user,count(sku),sum(price),      

   round(sum(price)/count(sku),2) as avgPrice from test where prince>5 group by user

  mapper=Code("""function (){emit(this.user,{amount:this.price,count:1,avgPrice:0})}""")

  reduce = Code("function (key, values) {"

    " var res={amount:0,count:0,avgPrice:0};"
    " for (var i = 0; i < values.length; i++) "
    " {"
      " res.count += values[i].count;"
      " res.amount += values[i].amount;"
    " }"
     " res.avgPrice = (res.amount/res.count).toFixed(2);"
    " return res;"
    "}")

  result = db.test.map_reduce(mapper,reduce,out ='myresults',query={'price':{'$gt': 6}})

   for doc in db.myresults.find():

    print doc

 

转载于:https://www.cnblogs.com/shaosks/p/5685354.html

你可能感兴趣的文章
项目申请文档提纲
查看>>
加密解密第二章:ollydbg用法
查看>>
百万PV网站架构
查看>>
N26-第四周作业
查看>>
在vmware安装Ubuntu桌面软件
查看>>
MySQL之用户和权限管理
查看>>
常用的命令的使用方法
查看>>
使用HeartBeat实现高可用HA的配置过程详解
查看>>
最常用的四种大数据分析方法
查看>>
ajax https请求返回json数据
查看>>
convenience - 便利构造函数
查看>>
golang 碎片整理之 结构体
查看>>
查看oracle查看当前连接以及修改最大连接数
查看>>
docker安装mysql镜像
查看>>
java中的IO整理
查看>>
我的linux学习决心书
查看>>
python 之多线程加锁
查看>>
我的友情链接
查看>>
exchange快速将断开的邮箱显示出来
查看>>
linux 下查找文件或者内容常用命令
查看>>