SmartNews is a news app with 20 million monthly active users in the U.S. and Japan. We are a machine-learning company deeply committed to helping users find quality news beyond the filter bubble.
Our algorithms evaluate tens of millions of articles, behaviors and social interactions to deliver only the most interesting and important stories on a global scale — a fascinating engineering challenge that has attracted a world-class data science and engineering team.
Join our international culture — smart work, smart people and a global mission to bring quality information to the world. Our perks include offering you a free healthy lunch, delicious coffee provided by our in-house barista, and financial support towards language-learning.
Your ideas matter
We view engineering as a collaborative activity, where our engineers aren’t only responsible for implementation, but also helping to figure out what we should build. So rather than just executing on a fully formed spec, you’ll work directly with product managers to come up with the best solutions from both a technical and product perspective. Ideas for new features or improvements don’t only flow down from above either, and so as an engineer, you’ll be encouraged to propose your own ideas for our product as well.
Not too big, not too small
We’re small enough that you can have a big impact on our product, but big enough that we’re tackling interesting and challenging problems at scale. We still don’t have much bureaucracy, nor an excessive amount of interdependencies between teams, so it’s still possible to go from idea to launch in very little time. For example, we launched our COVID-19 dashboard just 48 hours after it was proposed by an engineer on Slack.
Work both with your engineering peers and cross-functionally
Most companies organize their teams either by grouping people of a similar skillset, or assigning them to cross functional teams. We do both! You’ll be part of a “pillar”, where you’ll work with other engineers with a similar skillset, letting you trade ideas and level up each other’s technical skills, and also a “squad”, where you’ll work with a diverse cross-functional team to directly contribute to business goals.
An international environment that doesn’t require Japanese
Over fifty percent of our engineers are non-Japanese, coming from over 15 different countries. Although we don’t have an official language, English is becoming the de-facto standard of the engineering team. When needed, we have a team of interpreters that can provide simultaneous translation at meetings. Of course, if you’re looking to learn or improve Japanese, there’ll be plenty of opportunities to do so, including through free language lessons.
We welcome people who are currently overseas but looking to relocate to Japan. We can provide visa support and help you get set up in Japan.
Our response to COVID-19
Our user base is increasing quickly due to people’s interest in news (COVID and non-COVID related), and so we’re still hiring. If you’re abroad looking to relocate to Japan, we’re still interested. If you’re based in China or the US, we can initially hire you with our local branches, and then transfer you here once your visa is processed. If you’re based in another location, we’ll bring you once we’ve helped you obtain a visa.
For our existing employees, in response to the situation, we’re having everyone work remotely for the time being.
One specific challenge we’ve noticed remote work creates is that it’s hard to have the kind of spontaneous conversations you might have when grabbing a coffee in an office environment. To encourage our employees to meet other employees they might not know, and also catch up with those they’ve worked with in the past, we have created a bot that on a weekly basis randomly assigns employees to a small group for a short casual chat.
Our hiring process
When you submit your application to SmartNews, it will be reviewed by our recruitment team, to make sure it is relevant to the position. When you pass this initial screening, you’ll then be sent out an online coding test. The results of this test and your resume will then be reviewed in more detail by one of our engineers. After you pass this secondary screening, you’ll be invited in for a round of interviews, covering both behavioral and technical topics. Once you’ve passed these interviews, we’ll make you an offer.
About the position
The data platform software engineer in SmartNews plays a key role in accelerating products/business development. Great efforts are made on building a highly efficient and flexible data service for analytical and operational purposes.
To serve the internal users such as our analytics and product-dev teams, the mission of data engineers is to create high-level, easy-to-use data services for simplifying the accessing, integration, and consolidation of various data sets, and also building the platforms for executing tasks processing massive data in terms of TB per day.
Technology drives the growth of SmartNews, and thus we eagerly adopt cutting-edge technologies from the industry and academia, especially the open-source community.
- Design, develop, set up, and maintain new services, libraries, tools, frameworks for data processing or management, and investigate new algorithms to increase efficiency for Data Processing, such as ETL, Data Pipelines, OLAP DBMS, real-time messages and streams processing, data-sync between systems, etc.
- Develop and operate in-housing data service offerings such as high performant K/V store, adopt/enhance open-source frameworks to meet SmartNews’ growing business needs
- Develop tooling for system performance evaluation, monitoring, and tuning of the data processing procedures or platforms, get insights of efficiency and stabilizability and make continuous improvements, such as optimizing distributed query engines, computing resource management and isolation, multi-tier storage systems, etc.
- Own and maintain the key data processing portfolios such as building and taking care of the environment, troubleshooting, and being responsible for incidents that occur during on-call periods. Work closely with data architecting/modeling roles to understand ways to implement the data service, and interact with the Site Reliability Engineering (SRE) team to deploy the environments and drive production excellence.
- Devise system, tooling, and approaches for data privacy and security. Establish access control, create processes to handle sensitive data.
- Diagnose and resolve complex technical challenges for data accessing or processing. Using elegant and systematic rather than ad-hoc methods to help other teams tune the performance and improve stability.
- BS/MS degree in computer science or equivalent science/engineering degree with 5 or more years of experience
- Strong Programming skills and experiences with a deep understanding of data structures and algorithms are required for building efficient and stable solutions
- Rich experiences with one or more programming languages such as Java, Scala, C++, or Python; familiar with agile development and manage testing skills
- Need certain knowledge on shell scripts and operating systems, especially on Linux
- Good understanding of modern bigdata technologies and ecosystems
- Familiar with Hadoop, Spark, Hive, Presto, Redis, and Flink, be able to develop data processing programs with them in batch or streaming manner
- Familiar with modern data stores either RDBMS or NoSQL stores (such as MySQL, Cassandra or Druid, etc); have experience with developing application or function-extensions on such data stores
- Be able to implement and tune complicated heavy-lifting data flows (ETLs or pipelines), familiar with certain toolings
- Capability of system design with good modularity and extensibility
- Familiar with system/module design methods and toolings such as UML
- Be able to draft the user-understandable blueprint and precise, detailed designs
- Experience with building highly scalable distributed systems
- Able to design and implement distributed services with scalability and performance in mind
- Able to debug and troubleshoot performance and reliability problems