Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于网站埋点捕获点击事件的疑问 #7

Open
derekng622 opened this issue Oct 11, 2018 · 2 comments
Open

关于网站埋点捕获点击事件的疑问 #7

derekng622 opened this issue Oct 11, 2018 · 2 comments

Comments

@derekng622
Copy link

derekng622 commented Oct 11, 2018

ZzXxL1994你好,你在项目描述说: 通过在电影网站系统埋点,获取到用户的点击事件(如用户喜欢哪部电影或对某部电影的评分)并将信息传至推荐系统,推荐系统根据该信息做出相应的处理。

nginx.conf配置文件的疑问

nginx.conf文件

log_format my_format '$args';
...
       location = /log.gif {
  	    default_type image/gif;
   	    access_log /opt/movieLog/access.log my_format;
	}

问题1:请问这里匹配的uri是/log.gif是记录用户的什么请求?$args是什么参数?

流处理类SparkDrStreamALS.scala中处理日志的疑问

SparkDrStreamALS.scala

val messages = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topics)
val messagesDStream = messages.map(_._2).map(line => {
      var rating: UserRating = null
      if (line != null) {
        try {
          val fields = line.split(",")
          if (fields.length == 3) {
            rating = UserRating(fields(0).toInt, fields(1).toInt, fields(2).toDouble)
          }
        } catch {
          case e => e.printStackTrace()
        }
      }
      rating
    })

问题二:按照nginx配置的log_format的$args ,这个变量等于请求行中的参数,日志行的内容类似p=boo&q=foo,请问通过KafkaUtils.createDirectStream后的message RDD是怎样子的?后面使用message.map(_._2).map(line=>{…}应该是取得message的第二个元组作为line,请问是kafka的api自动将“&”符号映射为“,”逗号作为分隔符的吗?或是flume做了转换了吗?但是在flume*.properties未看到相关配置,只看到tail -F /opt/movieLog/access.log

@zhouke1991
Copy link

/log.gif应该是嵌码文件中发出的,防止跨域而使用的gif后缀,那么请问嵌码文件在哪可以看到?

@luochana
Copy link

问题二同问

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants