초간단 nasdaq 데이터분석

dev/NoSQL 모델링2017. 4. 27. 21:40

뷰어
댓글로
이전글
다음글

[mongodb에서 Aggregation 방법]

	group	map / reduce	aggregate f/w
map	X	O	△
reduce	O	O	O
분산집계 with shard	X	O	O
실행결과	document	-out collection - document	document
용도	online	batch	o / b

myung@JMYUNG C:\Users\myung
> mongod.exe --dbpath c:\data\db --port 27017

1. 복구

mongodump

- BSON형태로 저장

- 백업용도

- mongorestore로 복구

myung@JMYUNG C:\Users\myung
> mongorestore --host localhost c:\nasdaq_sample
2017-05-09T15:47:07.454+0900    preparing collections to restore from
2017-05-09T15:47:07.566+0900    reading metadata for nasdaq.stocks from c:\nasdaq_sample\nasdaq\stocks.metadata.json
2017-05-09T15:47:07.566+0900    reading metadata for nasdaq.symbols from c:\nasdaq_sample\nasdaq\symbols.metadata.json
2017-05-09T15:47:07.586+0900    restoring nasdaq.stocks from c:\nasdaq_sample\nasdaq\stocks.bson
2017-05-09T15:47:07.605+0900    restoring nasdaq.symbols from c:\nasdaq_sample\nasdaq\symbols.bson
2017-05-09T15:47:07.707+0900    no indexes to restore
2017-05-09T15:47:07.707+0900    finished restoring nasdaq.symbols (5960 documents)
2017-05-09T15:47:10.446+0900    [#.......................] nasdaq.stocks 33.0MB/715MB (4.6%)
2017-05-09T15:47:13.445+0900    [##......................] nasdaq.stocks 72.7MB/715MB (10.2%)
2017-05-09T15:47:16.447+0900    [###.....................] nasdaq.stocks 107MB/715MB (15.0%)
2017-05-09T15:47:19.445+0900    [####....................] nasdaq.stocks 145MB/715MB (20.3%)
2017-05-09T15:47:22.446+0900    [######..................] nasdaq.stocks 181MB/715MB (25.4%)
2017-05-09T15:47:25.446+0900    [#######.................] nasdaq.stocks 219MB/715MB (30.6%)
2017-05-09T15:47:28.445+0900    [########................] nasdaq.stocks 256MB/715MB (35.8%)
2017-05-09T15:47:31.446+0900    [#########...............] nasdaq.stocks 290MB/715MB (40.6%)
2017-05-09T15:47:34.446+0900    [##########..............] nasdaq.stocks 327MB/715MB (45.7%)
2017-05-09T15:47:37.453+0900    [############............] nasdaq.stocks 358MB/715MB (50.1%)
2017-05-09T15:47:40.446+0900    [#############...........] nasdaq.stocks 390MB/715MB (54.5%)
2017-05-09T15:47:43.446+0900    [##############..........] nasdaq.stocks 424MB/715MB (59.2%)
2017-05-09T15:47:46.445+0900    [###############.........] nasdaq.stocks 462MB/715MB (64.6%)
2017-05-09T15:47:49.445+0900    [################........] nasdaq.stocks 498MB/715MB (69.6%)
2017-05-09T15:47:52.445+0900    [##################......] nasdaq.stocks 537MB/715MB (75.1%)
2017-05-09T15:47:55.446+0900    [###################.....] nasdaq.stocks 572MB/715MB (80.0%)
2017-05-09T15:47:58.445+0900    [####################....] nasdaq.stocks 610MB/715MB (85.3%)
2017-05-09T15:48:01.445+0900    [#####################...] nasdaq.stocks 642MB/715MB (89.8%)
2017-05-09T15:48:04.450+0900    [######################..] nasdaq.stocks 679MB/715MB (94.9%)
2017-05-09T15:48:07.446+0900    [#######################.] nasdaq.stocks 708MB/715MB (99.0%)
2017-05-09T15:48:08.030+0900    [########################] nasdaq.stocks 715MB/715MB (100.0%)
2017-05-09T15:48:08.031+0900    no indexes to restore
2017-05-09T15:48:08.031+0900    finished restoring nasdaq.stocks (4308303 documents)
2017-05-09T15:48:08.031+0900    done

2. 조회

> db

nasdaq

> use nasdaq
switched to db nasdaq

> show collections
stocks
symbols

collection

- rdb의 테이블

- 스키마없음



> db.users.insert({ userid:"myung" , name:"myung je sang"})
WriteResult({ "nInserted" : 1 })
> db.user.find()
> db.users.find()
{ "_id" : ObjectId("59116e3b52ae922ad8a03686"), "userid" : "myung", "name" : "myung je sang" }
> db.users.insert({ userid:"myung" , name:"myung je sang"})
WriteResult({ "nInserted" : 1 })
> db.users.find()
{ "_id" : ObjectId("59116e3b52ae922ad8a03686"), "userid" : "myung", "name" : "myung je sang" }
{ "_id" : ObjectId("59116e5152ae922ad8a03687"), "userid" : "myung", "name" : "myung je sang" }
> db.users.insert({ userid:"myung2" , name:"myung je sang", gender:"man"})
WriteResult({ "nInserted" : 1 })
> db.users.find()
{ "_id" : ObjectId("59116e3b52ae922ad8a03686"), "userid" : "myung", "name" : "myung je sang" }
{ "_id" : ObjectId("59116e5152ae922ad8a03687"), "userid" : "myung", "name" : "myung je sang" }
{ "_id" : ObjectId("59116e6952ae922ad8a03688"), "userid" : "myung2", "name" : "myung je sang", "gender" : "man" }
> db.users.insert({ _id:1, userid:"myung3" , name:"myung je sang", gender:"man"})
WriteResult({ "nInserted" : 1 })
> db.users.find()
{ "_id" : ObjectId("59116e3b52ae922ad8a03686"), "userid" : "myung", "name" : "myung je sang" }
{ "_id" : ObjectId("59116e5152ae922ad8a03687"), "userid" : "myung", "name" : "myung je sang" }
{ "_id" : ObjectId("59116e6952ae922ad8a03688"), "userid" : "myung2", "name" : "myung je sang", "gender" : "man" }
{ "_id" : 1, "userid" : "myung3", "name" : "myung je sang", "gender" : "man" }
> db.users.insert({ _id:1, userid:"myung3" , name:"myung je sang", gender:"man"})
WriteResult({
        "nInserted" : 0,
        "writeError" : {
                "code" : 11000,
                "errmsg" : "E11000 duplicate key error collection: test.users index: _id_ dup key: { : 1.0 }"
        }
})
> db.users.save({ _id:1, userid:"myung3" , name:"myung je sang", gender:"man"})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 0 })
> db.users.save({ _id:1, userid:"myung4" , name:"myung je sang", gender:"man"})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.users.find()
{ "_id" : ObjectId("59116e3b52ae922ad8a03686"), "userid" : "myung", "name" : "myung je sang" }
{ "_id" : ObjectId("59116e5152ae922ad8a03687"), "userid" : "myung", "name" : "myung je sang" }
{ "_id" : ObjectId("59116e6952ae922ad8a03688"), "userid" : "myung2", "name" : "myung je sang", "gender" : "man" }
{ "_id" : 1, "userid" : "myung4", "name" : "myung je sang", "gender" : "man" }

- 컬렉션단위의 조회성능이 더 좋음

- 인덱싱은 컬렉션 단위

> db.stocks.find()
{ "_id" : ObjectId("4d094f58c96767d7a0099d49"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-07", "open" : 8.4, "high" : 8.75, "low" : 8.08, "close" : 8.55, "volume" : 275800, "adj close" : 8.55 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4a"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-06", "open" : 9.03, "high" : 9.03, "low" : 8.41, "close" : 8.56, "volume" : 353600, "adj close" : 8.56 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4b"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-05", "open" : 9.12, "high" : 9.17, "low" : 8.85, "close" : 9.12, "volume" : 156200, "adj close" : 9.12 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4c"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-04", "open" : 9.05, "high" : 9.14, "low" : 8.73, "close" : 9.09, "volume" : 420700, "adj close" : 9.09 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4d"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-03", "open" : 9.68, "high" : 9.69, "low" : 8.98, "close" : 9.15, "volume" : 407200, "adj close" : 9.15 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4e"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-29", "open" : 9.52, "high" : 9.76, "low" : 9.25, "close" : 9.75, "volume" : 269400, "adj close" : 9.75 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4f"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-28", "open" : 9.7, "high" : 10.1, "low" : 9.67, "close" : 9.7, "volume" : 150200, "adj close" : 9.7 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d50"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-27", "open" : 9.8, "high" : 10.25, "low" : 9.58, "close" : 9.76, "volume" : 190700, "adj close" : 9.76 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d51"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-26", "open" : 9.4, "high" : 9.94, "low" : 9.25, "close" : 9.91, "volume" : 205100, "adj close" : 9.91 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d52"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-25", "open" : 9.84, "high" : 10, "low" : 9.37, "close" : 9.79, "volume" : 352200, "adj close" : 9.79 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d53"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-22", "open" : 10.19, "high" : 10.19, "low" : 9.75, "close" : 10, "volume" : 305400, "adj close" : 10 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d54"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-21", "open" : 10.42, "high" : 10.53, "low" : 10.25, "close" : 10.42, "volume" : 126100, "adj close" : 10.42 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d55"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-20", "open" : 9.91, "high" : 10.49, "low" : 9.91, "close" : 10.33, "volume" : 99700, "adj close" : 10.33 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d56"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-19", "open" : 10.31, "high" : 10.4, "low" : 9.94, "close" : 10.01, "volume" : 170100, "adj close" : 10.01 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d57"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-15", "open" : 10.08, "high" : 10.29, "low" : 9.93, "close" : 10.15, "volume" : 259700, "adj close" : 10.15 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d58"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-14", "open" : 9.98, "high" : 10.84, "low" : 9.98, "close" : 10.13, "volume" : 461300, "adj close" : 10.13 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d59"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-13", "open" : 9.64, "high" : 9.98, "low" : 9.57, "close" : 9.98, "volume" : 78700, "adj close" : 9.98 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d5a"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-12", "open" : 9.53, "high" : 9.62, "low" : 9.46, "close" : 9.53, "volume" : 60500, "adj close" : 9.53 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d5b"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-11", "open" : 9.51, "high" : 9.62, "low" : 9.17, "close" : 9.46, "volume" : 225800, "adj close" : 9.46 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d5c"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-08", "open" : 9.87, "high" : 9.99, "low" : 9.48, "close" : 9.49, "volume" : 114800, "adj close" : 9.49 }

> db.stocks.find({"stock_symbol": "GOOG"}).sort({date:1}).explain("executionStats")
{
        "queryPlanner" : {
                "plannerVersion" : 1,
                "namespace" : "nasdaq.stocks",
                "indexFilterSet" : false,
                "parsedQuery" : {
                        "stock_symbol" : {
                                "$eq" : "GOOG"
                        }
                },
                "winningPlan" : {
                        "stage" : "SORT",
                        "sortPattern" : {
                                "date" : 1
                        },
                        "inputStage" : {
                                "stage" : "SORT_KEY_GENERATOR",
                                "inputStage" : {
                                        "stage" : "COLLSCAN",
                                        "filter" : {
                                                "stock_symbol" : {
                                                        "$eq" : "GOOG"
                                                }
                                        },
                                        "direction" : "forward"
                                }
                        }
                },
                "rejectedPlans" : [ ]
        },
        "executionStats" : {
                "executionSuccess" : true,
                "nReturned" : 894,
                "executionTimeMillis" : 2051,
                "totalKeysExamined" : 0,
                "totalDocsExamined" : 4308303,
                "executionStages" : {
                        "stage" : "SORT",
                        "nReturned" : 894,
                        "executionTimeMillisEstimate" : 1711,
                        "works" : 4309201,
                        "advanced" : 894,
                        "needTime" : 4308306,
                        "needYield" : 0,
                        "saveState" : 33710,
                        "restoreState" : 33710,
                        "isEOF" : 1,
                        "invalidates" : 0,
                        "sortPattern" : {
                                "date" : 1
                        },
                        "memUsage" : 163584,
                        "memLimit" : 33554432,
                        "inputStage" : {
                                "stage" : "SORT_KEY_GENERATOR",
                                "nReturned" : 894,
                                "executionTimeMillisEstimate" : 1650,
                                "works" : 4308306,
                                "advanced" : 894,
                                "needTime" : 4307411,
                                "needYield" : 0,
                                "saveState" : 33710,
                                "restoreState" : 33710,
                                "isEOF" : 1,
                                "invalidates" : 0,
                                "inputStage" : {
                                        "stage" : "COLLSCAN",
                                        "filter" : {
                                                "stock_symbol" : {
                                                        "$eq" : "GOOG"
                                                }
                                        },
                                        "nReturned" : 894,
                                        "executionTimeMillisEstimate" : 1608,
                                        "works" : 4308305,
                                        "advanced" : 894,
                                        "needTime" : 4307410,
                                        "needYield" : 0,
                                        "saveState" : 33710,
                                        "restoreState" : 33710,
                                        "isEOF" : 1,
                                        "invalidates" : 0,
                                        "direction" : "forward",
                                        "docsExamined" : 4308303
                                }
                        }
                }
        },
        "serverInfo" : {
                "host" : "jmyung",
                "port" : 27017,
                "version" : "3.4.4",
                "gitVersion" : "888390515874a9debd1b6c5d36559ca86b44babd"
        },
        "ok" : 1
}

------------------------------------------------------------------------------------------------------

> db.stocks.count()
4308303
> db.stocks.find()
{ "_id" : ObjectId("4d094f58c96767d7a0099d49"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-07", "open" : 8.4, "high" : 8.75, "low" : 8.08, "close" : 8.55, "volume" : 275800, "adj close" : 8.55 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4a"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-06", "open" : 9.03, "high" : 9.03, "low" : 8.41, "close" : 8.56, "volume" : 353600, "adj close" : 8.56 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4b"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-05", "open" : 9.12, "high" : 9.17, "low" : 8.85, "close" : 9.12, "volume" : 156200, "adj close" : 9.12 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4c"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-04", "open" : 9.05, "high" : 9.14, "low" : 8.73, "close" : 9.09, "volume" : 420700, "adj close" : 9.09 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4d"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-03-03", "open" : 9.68, "high" : 9.69, "low" : 8.98, "close" : 9.15, "volume" : 407200, "adj close" : 9.15 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4e"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-29", "open" : 9.52, "high" : 9.76, "low" : 9.25, "close" : 9.75, "volume" : 269400, "adj close" : 9.75 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d4f"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-28", "open" : 9.7, "high" : 10.1, "low" : 9.67, "close" : 9.7, "volume" : 150200, "adj close" : 9.7 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d50"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-27", "open" : 9.8, "high" : 10.25, "low" : 9.58, "close" : 9.76, "volume" : 190700, "adj close" : 9.76 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d51"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-26", "open" : 9.4, "high" : 9.94, "low" : 9.25, "close" : 9.91, "volume" : 205100, "adj close" : 9.91 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d52"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-25", "open" : 9.84, "high" : 10, "low" : 9.37, "close" : 9.79, "volume" : 352200, "adj close" : 9.79 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d53"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-22", "open" : 10.19, "high" : 10.19, "low" : 9.75, "close" : 10, "volume" : 305400, "adj close" : 10 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d54"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-21", "open" : 10.42, "high" : 10.53, "low" : 10.25, "close" : 10.42, "volume" : 126100, "adj close" : 10.42 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d55"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-20", "open" : 9.91, "high" : 10.49, "low" : 9.91, "close" : 10.33, "volume" : 99700, "adj close" : 10.33 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d56"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-19", "open" : 10.31, "high" : 10.4, "low" : 9.94, "close" : 10.01, "volume" : 170100, "adj close" : 10.01 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d57"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-15", "open" : 10.08, "high" : 10.29, "low" : 9.93, "close" : 10.15, "volume" : 259700, "adj close" : 10.15 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d58"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-14", "open" : 9.98, "high" : 10.84, "low" : 9.98, "close" : 10.13, "volume" : 461300, "adj close" : 10.13 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d59"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-13", "open" : 9.64, "high" : 9.98, "low" : 9.57, "close" : 9.98, "volume" : 78700, "adj close" : 9.98 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d5a"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-12", "open" : 9.53, "high" : 9.62, "low" : 9.46, "close" : 9.53, "volume" : 60500, "adj close" : 9.53 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d5b"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-11", "open" : 9.51, "high" : 9.62, "low" : 9.17, "close" : 9.46, "volume" : 225800, "adj close" : 9.46 }
{ "_id" : ObjectId("4d094f58c96767d7a0099d5c"), "exchange" : "NASDAQ", "stock_symbol" : "AACC", "date" : "2008-02-08", "open" : 9.87, "high" : 9.99, "low" : 9.48, "close" : 9.49, "volume" : 114800, "adj close" : 9.49 }
Type "it" for more
> db.stocks.count({stock_symbol:"AACC"})
1016
> db.stocks.count({stock_symbol:"AACC", close:{$lt: 300}})
1016
> db.stocks.count({stock_symbol:"GOOG", close:{$lt: 300}})
262
> db.stocks.find({stock_symbol:"GOOG", close:{$lt: 300}})
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d4f"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-10-14", "open" : 299.9, "high" : 300.23, "low" : 292.54, "close" : 296.14, "volume" : 8519100, "adj close" : 296.14 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d50"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-10-13", "open" : 302, "high" : 302, "low" : 290.68, "close" : 297.44, "volume" : NumberLong(10567700), "adj close" : 297.44 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d68"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-09-09", "open" : 297.28, "high" : 299.1, "low" : 296.56, "close" : 299.09, "volume" : 4390500, "adj close" : 299.09 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d69"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-09-08", "open" : 294.83, "high" : 299.28, "low" : 293.36, "close" : 295.39, "volume" : 6613300, "adj close" : 295.39 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d6a"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-09-07", "open" : 285.89, "high" : 295.5, "low" : 285.28, "close" : 294.87, "volume" : 7499500, "adj close" : 294.87 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d6b"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-09-06", "open" : 289, "high" : 289.39, "low" : 286.8, "close" : 287.11, "volume" : 4212300, "adj close" : 287.11 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d6c"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-09-02", "open" : 286.51, "high" : 289.99, "low" : 286.44, "close" : 288.45, "volume" : 3434500, "adj close" : 288.45 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d6d"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-09-01", "open" : 285.91, "high" : 287.5, "low" : 285, "close" : 286.25, "volume" : 2742100, "adj close" : 286.25 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d6e"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-31", "open" : 288.23, "high" : 288.5, "low" : 284.36, "close" : 286, "volume" : 5034000, "adj close" : 286 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d6f"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-30", "open" : 287.39, "high" : 289.51, "low" : 285.88, "close" : 287.27, "volume" : 4792000, "adj close" : 287.27 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d70"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-29", "open" : 282.24, "high" : 289.12, "low" : 282.24, "close" : 288.45, "volume" : 5903000, "adj close" : 288.45 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d71"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-26", "open" : 283.48, "high" : 285.02, "low" : 282.66, "close" : 283.58, "volume" : 3755300, "adj close" : 283.58 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d72"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-25", "open" : 282.55, "high" : 284, "low" : 279.97, "close" : 282.59, "volume" : 4376600, "adj close" : 282.59 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d73"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-24", "open" : 277.57, "high" : 284.75, "low" : 276.45, "close" : 282.57, "volume" : 8593100, "adj close" : 282.57 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d74"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-23", "open" : 276.16, "high" : 279.74, "low" : 274.12, "close" : 279.58, "volume" : 5821700, "adj close" : 279.58 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d75"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-22", "open" : 281.24, "high" : 281.47, "low" : 273.35, "close" : 274.01, "volume" : 6813000, "adj close" : 274.01 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d76"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-19", "open" : 280.99, "high" : 281.45, "low" : 279.62, "close" : 280, "volume" : 5542900, "adj close" : 280 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d77"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-18", "open" : 275.91, "high" : 280.5, "low" : 275, "close" : 279.99, "volume" : NumberLong(11872800), "adj close" : 279.99 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d78"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-17", "open" : 285.51, "high" : 286.57, "low" : 284, "close" : 285.1, "volume" : 3883300, "adj close" : 285.1 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0d79"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2005-08-16", "open" : 284.88, "high" : 287.79, "low" : 283.34, "close" : 285.65, "volume" : 7109200, "adj close" : 285.65 }
Type "it" for more

> db.stocks.find({stock_symbol:"GOOG", close:{$gt: 300}})
{ "_id" : ObjectId("4d094f7ec96767d7a02a0af6"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-03-07", "open" : 428.88, "high" : 440, "low" : 426.24, "close" : 433.35, "volume" : 8071800, "adj close" : 433.35 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0af7"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-03-06", "open" : 447.69, "high" : 453.3, "low" : 431.18, "close" : 432.7, "volume" : 7470100, "adj close" : 432.7 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0af8"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-03-05", "open" : 445.25, "high" : 454.17, "low" : 444, "close" : 447.7, "volume" : 7436600, "adj close" : 447.7 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0af9"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-03-04", "open" : 450.95, "high" : 453.36, "low" : 435.78, "close" : 444.6, "volume" : NumberLong(13621700), "adj close" : 444.6 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0afa"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-03-03", "open" : 471.51, "high" : 472.72, "low" : 450.11, "close" : 457.02, "volume" : 7554500, "adj close" : 457.02 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0afb"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-29", "open" : 471.87, "high" : 479.74, "low" : 464.65, "close" : 471.18, "volume" : 9425400, "adj close" : 471.18 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0afc"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-28", "open" : 470.5, "high" : 479.09, "low" : 467.36, "close" : 475.39, "volume" : 6586900, "adj close" : 475.39 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0afd"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-27", "open" : 460.13, "high" : 475.49, "low" : 459.64, "close" : 472.86, "volume" : NumberLong(10121900), "adj close" : 472.86 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0afe"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-26", "open" : 461.2, "high" : 466.47, "low" : 446.85, "close" : 464.19, "volume" : NumberLong(23287300), "adj close" : 464.19 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0aff"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-25", "open" : 505.95, "high" : 506.5, "low" : 485.74, "close" : 486.44, "volume" : 8350800, "adj close" : 486.44 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b00"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-22", "open" : 502.06, "high" : 509, "low" : 497.55, "close" : 507.8, "volume" : 5515900, "adj close" : 507.8 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b01"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-21", "open" : 512.85, "high" : 513.21, "low" : 499.5, "close" : 502.86, "volume" : 5677800, "adj close" : 502.86 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b02"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-20", "open" : 503.51, "high" : 511.01, "low" : 498.82, "close" : 509, "volume" : 6662200, "adj close" : 509 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b03"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-19", "open" : 534.94, "high" : 535.06, "low" : 506.5, "close" : 508.95, "volume" : 6350400, "adj close" : 508.95 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b04"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-15", "open" : 528.31, "high" : 532.66, "low" : 524.33, "close" : 529.64, "volume" : 5240100, "adj close" : 529.64 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b05"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-14", "open" : 538.35, "high" : 541.04, "low" : 531, "close" : 532.25, "volume" : 6476700, "adj close" : 532.25 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b06"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-13", "open" : 522.5, "high" : 534.99, "low" : 518.69, "close" : 534.62, "volume" : 6624700, "adj close" : 534.62 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b07"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-12", "open" : 523.39, "high" : 530.6, "low" : 513.03, "close" : 518.09, "volume" : 6662300, "adj close" : 518.09 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b08"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-11", "open" : 520.52, "high" : 523.71, "low" : 513.4, "close" : 521.16, "volume" : 5826000, "adj close" : 521.16 }
{ "_id" : ObjectId("4d094f7ec96767d7a02a0b09"), "exchange" : "NASDAQ", "stock_symbol" : "GOOG", "date" : "2008-02-08", "open" : 509.41, "high" : 517.73, "low" : 508.7, "close" : 516.69, "volume" : 6828900, "adj close" : 516.69 }
Type "it" for more
>

3. 분석

db.stocks.distinct("stock_symbol", {close: {$gt:300}})

GO로 시작하는 종목명의 2006년 한해동안 평균종가와 최고종가

3-1. group함수이용

10000개 이상 처리 안됨 (10000개 미만은 m/r보다 좋은 선택)

> db.stocks.group({
... key : { stock_symbol : true },
... cond : { stock_symbol : /^GO/,
...          date : {$gte : '2006-01-01', $lte : '2016-12-31'}
...        },
... reduce : function(doc,prev) {
...        if (doc.close > prev.maxClose)
...           prev.maxClose = doc.close;
...        prev.sum += doc.close;
...        prev.count++;
... },
... initial : {sum:0, count:0, maxClose:0},
... finalize : function(prev) {
...        prev.avg = prev.sum / prev.count;
...        prev.max = prev.maxClose;
...        delete prev.maxClose;
...        delete prev.sum;
...        delete prev.count;
... }
... })
[
        {
                "stock_symbol" : "GOAM",
                "avg" : 5.132883211678831,
                "max" : 10.3
        },
        {
                "stock_symbol" : "GOLD",
                "avg" : 25.65408759124087,
                "max" : 55.65
        },
        {
                "stock_symbol" : "GOLF",
                "avg" : 7.4655319148936155,
                "max" : 11.49
        },
        {
                "stock_symbol" : "GOOD",
                "avg" : 19.049215328467145,
                "max" : 22.19
        },
        {
                "stock_symbol" : "GOODO",
                "avg" : 21.802142857142854,
                "max" : 22.45
        },
        {
                "stock_symbol" : "GOODP",
                "avg" : 24.28254641909813,
                "max" : 26.81
        },
        {
                "stock_symbol" : "GOOG",
                "avg" : 480.9729379562046,
                "max" : 741.79
        },
        {
                "stock_symbol" : "GORX",
                "avg" : 3.8033941605839385,
                "max" : 4.88
        }
]

저작자표시

'dev > NoSQL 모델링' 카테고리의 다른 글

mongodb GUI client (0)	2017.05.09
정리 (0)	2017.01.26
카산드라 (0)	2017.01.25
NoSql 모델링 기법 (0)	2017.01.25
Shard Cluster (1)	2017.01.25

Learn and Be Curious

초간단 nasdaq 데이터분석

'dev > NoSQL 모델링' 카테고리의 다른 글

최근에 올라온 글

최근에 달린 댓글

공지사항

글 보관함

링크

티스토리툴바

« 2024/07 »
일	월	화	수	목	금	토
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28	29	30	31